Evaluation system for college student occupational scene simulation training

Abstract

The invention relates to the technical field of occupational simulation training evaluation, and discloses a system for college student occupational scene simulation training evaluation. The system comprises a task analysis unit, a dynamic behavior modeling unit, a multi-mode interaction management unit, a state evolution tracking unit, a strategy adaptation unit and a comprehensive evaluation generation unit. The system receives the initial scene description and generates a structured task framework, and behavior parameters are configured for student roles; constructing a multi-modal interaction atlas by fusing aligned voice, action and sight line data in real time; identifying a key decision node, and dynamically tracking a team state evolution path; matching and adjusting strategies in a behavior rule base according to the evolution sequence, and outputting an adaptive intervention scheme; and finally generating a multi-dimensional structured evaluation report. According to the system, deep insight and dynamic and objective intelligent evaluation of complex interaction behaviors in the simulation training process are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of professional simulation training evaluation, in particular to a university student professional scene simulation training evaluation system. BACKGROUND

[0002] At present, the evaluation of university student professional scene simulation training mainly relies on on-site observation by instructors or post-event video playback. This evaluation method is highly dependent on personal experience and has strong subjectivity, and it is difficult to systematically analyze dynamic and complex interactive behaviors in team collaboration. Existing technologies usually only focus on the final task results or explicit language communication, and cannot effectively capture and analyze non-verbal cues and their role in team dynamics, resulting in single evaluation dimensions and insufficient depth and accuracy of feedback.

[0003] In terms of technical application, some systems attempt to introduce speech recognition or motion capture devices, but various types of data are usually processed independently, lacking effective integration mechanisms. Multimodal data is in isolation, and accurate temporal correlation and feature-level fusion cannot be established. This data fragmentation prevents the system from restoring the true full picture of the interaction scene, severely restricting the ability to objectively and meticulously analyze the team collaboration process.

[0004] In terms of analysis models, existing automated evaluation methods are mostly based on predetermined scoring rules to perform static evaluation on results or key nodes. This type of method cannot continuously track how individual behaviors in the training process trigger dynamic evolution of team states, and lacks identification and characterization of "decision-impact" causal paths. Due to the inability to understand the evolution logic of team internal states, evaluation results often remain at the level of phenomenon description, making it difficult to reveal the underlying reasons affecting team effectiveness, and even more difficult to provide adaptive intervention guidance based on real-time dynamics during simulation. Existing technologies have limitations in dynamic analysis and depth of insight. SUMMARY

[0005] The purpose of the present application is to provide a university student professional scene simulation training evaluation system to solve the problems raised in the background art.

[0006] To achieve the above purpose, the present application provides a university student professional scene simulation training evaluation system, which comprises:

[0007] A task analysis unit for receiving an initial professional scene description, performing semantic decomposition on the initial professional scene description, extracting scene core task elements and role constraint conditions, and generating a structured task framework;

[0008] A dynamic behavior modeling unit for importing a preset student role configuration based on the structured task framework, assigning initial behavior parameters to each student role configuration, and constructing a dynamic behavior rule library based on historical training data.

[0009] a multi-modal interaction management unit configured to capture voice interaction data, action sequence data and gaze focus data between students in real time during a simulation training process, timestamp-align and feature-fuse the voice interaction data, action sequence data and gaze focus data, and generate a multi-modal interaction graph;

[0010] a state evolution tracking unit configured to identify a decision node of an individual student according to the multi-modal interaction graph, track a team state change path triggered by the decision node, and record a state evolution sequence;

[0011] a strategy adaptation unit configured to match a candidate strategy in the dynamic behavior rule library based on the state evolution sequence, adjust a weight of the candidate strategy according to a real-time training environment parameter, and output an adapted intervention strategy;

[0012] a comprehensive evaluation generation unit configured to integrate the state evolution sequence and the adapted intervention strategy, calculate a performance consistency index of an individual and a team in multiple dimensions, and generate a structured evaluation report.

[0013] Preferably, the semantic decomposition of the initial professional scene description comprises:

[0014] adopting a hierarchical semantic parsing method to split the initial professional scene description into a scene background section, a task objective section and a constraint condition section;

[0015] performing entity recognition and relationship extraction on the scene background section to establish a scene entity relationship network;

[0016] performing action verb analysis and logical dependency parsing on the task objective section to construct a task objective tree;

[0017] performing rule clause classification and priority assignment on the constraint condition section to form a constraint rule set;

[0018] associating and mapping the scene entity relationship network, the task objective tree and the constraint rule set to generate the structured task framework.

[0019] Preferably, the construction of the dynamic behavior rule library based on historical training data comprises:

[0020] extracting a typical behavior pattern segment from historical training logs, performing cluster analysis on the typical behavior pattern segment to form a behavior pattern template;

[0021] performing condition-action pair analysis on each behavior pattern template to identify a trigger condition set and a corresponding action set;

[0022] The trigger condition set and the corresponding action set are formally described using a state machine representation method to generate standardized behavior rules;

[0023] The standardized behavior rules are indexed and arranged according to the training scenario type to construct the dynamic behavior rule library.

[0024] Preferably, the step of aligning the voice interaction data, action sequence data, and gaze focus data with timestamps and fusing features includes:

[0025] Extract speech content features, intonation features, and speech rate features from the speech interaction data;

[0026] Extract joint motion trajectory features, motion amplitude features, and motion frequency features from the motion sequence data;

[0027] Extract gaze target features, gaze duration features, and saccade path features from the gaze focus data;

[0028] Establish a unified timeline and align the speech content features, intonation features, speech rate features, joint movement trajectory features, movement amplitude features, movement frequency features, gaze target features, gaze duration features, and saccade path features according to time windows;

[0029] An attention mechanism is used to perform weighted fusion of aligned multi-source features to generate the multimodal interaction map.

[0030] Preferably, tracking the team state change path triggered by the decision node includes:

[0031] Mark the decision-making entity and decision content at the decision-making node;

[0032] Simulate the execution of the aforementioned decision and predict the reaction behavior of other students in subsequent time steps;

[0033] The actual occurrence of the reaction behavior is verified based on the multimodal interaction graph, and the deviation between the actual behavior and the expected behavior is recorded.

[0034] The causal chain between the decision nodes, the actual reaction behaviors, and the deviations is recorded in a graph structure to form the state evolution sequence.

[0035] Preferably, the step of adjusting the weights of the candidate strategies based on real-time training environment parameters includes:

[0036] Real-time monitoring of task progress metrics and team collaboration metrics in training scenarios;

[0037] The task progress indicators are compared with preset milestones to calculate the progress deviation.

[0038] The team collaboration metrics are compared with the ideal collaboration model to calculate the collaboration efficiency value;

[0039] Based on the progress deviation and the collaboration efficiency value, the activation threshold and execution priority of the candidate strategies are dynamically adjusted.

[0040] Based on the adjusted activation threshold and execution priority, the appropriate intervention strategy is selected from the candidate strategies.

[0041] Preferably, the consistency index of individual and team performance across multiple dimensions includes:

[0042] Define the dimensions of task execution, communication and coordination, and emergency response;

[0043] Under the aforementioned task execution dimension, calculate the matching coefficient between individual task completion and team task synchronization.

[0044] Under the aforementioned communication and coordination dimension, the correlation index between individual communication initiative and team information flow efficiency is analyzed;

[0045] Under the aforementioned emergency response dimension, assess the coordination ratio between individual decision-making speed and team crisis recovery speed;

[0046] The matching coefficient, correlation index, and coordination ratio are normalized and weighted to generate the performance consistency index.

[0047] Preferably, the step of associating and mapping the scene entity relationship network, task target tree, and constraint rule set includes:

[0048] Identify key entity nodes in the scene entity relationship network, and perform attribute matching between the key entity nodes and the leaf nodes of the task target tree;

[0049] Establish the constraint binding relationship between the constraint rule set and the branch nodes in the task target tree;

[0050] When an attribute matching conflict or constraint binding conflict is detected, the conflict resolution protocol is initiated to reallocate entity attributes or adjust constraint priorities.

[0051] After the conflict is resolved, a structured task framework containing complete relationships between entities, tasks, and constraints is generated.

[0052] Preferably, the step of using an attention mechanism to perform weighted fusion of aligned multi-source features includes:

[0053] Calculate the relevance score of each feature dimension to the core task of the current training phase;

[0054] Initial attention weights are assigned based on the relevance scores;

[0055] Contextual information from the dynamic behavior rule base is incorporated to perform contextual correction on the initial attention weights;

[0056] The corresponding features are weighted and summed using the modified attention weights to generate a fused feature vector;

[0057] The fused feature vectors are organized into graph nodes according to the time series, and edges are connected according to the temporal dependencies between features to construct the multimodal interaction graph.

[0058] Preferably, the step of recording the causal chain between the decision node, the actual reaction behavior, and the deviation in a graph structure includes:

[0059] Each decision node is used as the root node of the cause-effect graph;

[0060] Connect the actual reaction behavior as a child node to the corresponding root node, and mark the behavior type and timestamp;

[0061] The deviation is labeled as an edge attribute on the edge connecting the root node and the child node, and the label content includes the deviation amount and deviation type.

[0062] When multiple decision nodes are temporally or logically related, cross-node connection edges are established between the corresponding root nodes to form the state evolution sequence that describes the state propagation path.

[0063] Compared with the prior art, the beneficial effects of the present invention are:

[0064] By precisely aligning timestamps and fusing features from voice, motion, and eye contact data, a unified multimodal interaction map is generated, solving the problem of data fragmentation in traditional assessments. This technology correlates behavioral signals from different modalities within a unified temporal framework, enabling the reconstruction of collaborative relationships between elements such as speech content, corresponding gestures, and eye contact. This deep fusion reveals interaction patterns that isolated data cannot capture. It enhances the dimensionality and precision of behavior capture, ensuring that the assessment of individual communication and collaboration abilities is based on objective and comprehensive data, thus improving the objectivity and granularity of the evaluation.

[0065] By automatically identifying key decision-making nodes from the interaction flow and tracing the resulting evolution of team states, dynamic causal analysis of the team collaboration process is achieved. This technology can depict the complete chain of how a specific proposal or behavior gradually influences the direction of team discussions, task progress, and collaborative atmosphere. Based on this evolutionary sequence, the system can understand the driving mechanisms of dynamic changes within the team. This transforms the evaluation mechanism from static outcome judgment to a dynamic insight into the process, accurately identifying key behaviors that effectively drive progress or lead to inefficiency, thus providing feedback with profound interpretability and guidance.

[0066] By dynamically matching and adjusting intervention strategies based on the state evolution sequence, a closed-loop linkage between evaluation and intervention is achieved. The system can filter scenario-matching candidate strategies from the rule base and perform real-time weight optimization based on the actual evolution trend of the team's state, outputting a highly adaptive intervention plan. This allows the system to provide evaluation not only after training but also formative intervention based on team dynamics during the process. It enhances the intelligence and real-time guidance capabilities of the training system, making the simulation environment a learning scenario that can adaptively respond and guide team interaction, rather than simply an assessment tool. Attached Figure Description

[0067] Figure 1 This is a schematic diagram illustrating the working principle of the college student career scenario simulation training and evaluation system described in this invention.

[0068] Figure 2 A flowchart for semantic decomposition work;

[0069] Figure 3 A flowchart illustrating the process of building a dynamic behavior rule base;

[0070] Figure 4 A statistical chart showing the distribution of average deviations at each decision node;

[0071] Figure 5 A graph showing the correlation between parameter adjustment and strategy effectiveness. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Please see Figure 1This invention provides a simulation training and evaluation system for college students' professional scenarios. The system includes: receiving an initial professional scenario description; a task parsing unit semantically decomposing the description, extracting core task elements and role constraints, and generating a structured task framework; a dynamic behavior modeling unit importing student role configurations based on the structured task framework, assigning initial behavior parameters to each role, and constructing a dynamic behavior rule base based on historical training data; a multimodal interaction management unit capturing students' voice interaction data, action sequence data, and gaze focus data in real time during simulation training, aligning these data with timestamps and fusing features to generate a multimodal interaction graph; a state evolution tracking unit identifying individual students' decision nodes based on the multimodal interaction graph, tracking the team state change path triggered by these decision nodes, and recording the state evolution sequence; a strategy adaptation unit matching candidate strategies from the dynamic behavior rule base based on the state evolution sequence, adjusting the weights of the candidate strategies according to real-time training environment parameters, and outputting an adapted intervention strategy; and a comprehensive evaluation generation unit integrating the state evolution sequence and the adapted intervention strategy, calculating consistency indicators of individual and team performance across multiple dimensions, and generating a structured evaluation report.

[0074] Example 1: See Figure 2 In a specific implementation, the task parsing unit performs semantic decomposition on the initial occupational scenario description. A hierarchical semantic parsing method is used to split the initial occupational scenario description into a scenario background segment, a task objective segment, and a constraint condition segment. The hierarchical semantic parsing method divides the text according to its natural paragraph structure and logical connectors. For the scenario background segment, entity recognition and relation extraction are performed. Entity recognition uses named entity recognition technology to identify entity categories such as people, organizations, locations, and tools. Relation extraction uses dependency parsing to determine the action, ownership, or spatiotemporal relationships between entities, establishing a scenario entity relation network. This network is stored in a graph structure, where nodes represent entities and are labeled with attribute tags, and edges represent relations and are labeled with relation types. For the task objective segment, action verb analysis and logical dependency parsing are performed. Action verb analysis extracts and classifies core predicate verbs from sentences. Logical dependency parsing analyzes conjunctions and word order to determine the sequential, conditional, or parallel relationships between subtasks, constructing a task objective tree. The task objective tree uses a tree-like data structure, where the root node represents the overall task objective, child nodes represent the decomposed subtasks, and edges connecting nodes are labeled with dependency types such as "preconditions" or "synchronous execution." The constraint condition section is classified into rule clauses and assigned priority values. The rule clauses are classified into types such as time constraints, resource constraints, and role constraints according to the constraint content. The priority values ​​are assigned numerical values ​​based on the importance of the constraint clauses and the severity of the consequences of violation, forming a constraint rule set. The constraint rule set is stored in the form of a list, and each rule contains the constraint type, applicable object, constraint condition, and priority value.

[0075] In practical implementation, the scenario entity relationship network, task target tree, and constraint rule set are associated and mapped. Key entity nodes in the scenario entity relationship network are identified; these are entities directly related to the achievement of the task target or possessing important attributes. Key entity nodes are then matched with the leaf nodes of the task target tree. This matching process compares the attribute labels of the entity nodes with the task requirements of the leaf nodes; for example, the "skill level" attribute of an entity node is compared with the "required skill" requirement of a leaf node. Constraint binding relationships are established between the constraint rule set and the branch nodes in the task target tree. Each rule in the constraint rule set is associated with a branch node in the task target tree affected by that rule; for example, a time constraint rule will be bound to a branch node with a deadline requirement. When attribute matching conflicts or constraint binding conflicts are detected, a conflict resolution protocol is initiated. Attribute matching conflicts occur when entity attributes cannot meet task requirements, while constraint binding conflicts occur when multiple constraint rules create contradictory requirements on the same branch node. The conflict resolution protocol resolves conflicts by reallocating entity attributes or adjusting constraint priorities. Reallocating entity attributes may involve modifying the entity's role or capability assignments, while adjusting constraint priorities involves reassessing the importance order of constraint rules based on the overall task objective. After conflict resolution is complete, a structured task framework containing complete entity-task-constraint relationships is generated. This structured task framework uses a unified data model to integrate all relationships between the scenario entity relationship network, the task objective tree, and the constraint rule set.

[0076] In some embodiments, the hierarchical semantic parsing method specifically includes sentence and word segmentation of the initial occupational scenario description text, matching the text paragraph structure using a predefined scenario template library containing typical paragraph structure patterns for various occupational scenarios. The entity recognition and relation extraction process uses a deep learning-based sequence labeling model, trained on a large amount of labeled text, capable of accurately identifying specific entity types and relation patterns in occupational scenarios. Action verb analysis and logical dependency resolution employ semantic role labeling technology, labeling the agent, patient, and time semantic roles of verbs in sentences to infer the logical connections between actions. Rule classification and priority assignment rely on a predefined constraint classification system and a weight allocation algorithm based on rule importance, considering the authority of the rule source and the potential impact of rule violations.

[0077] In some embodiments, the association mapping process is implemented through a mapping engine. The mapping engine traverses all entity nodes in the scene entity relationship network and calculates their semantic similarity with the leaf nodes of the task target tree. The semantic similarity calculation is based on a word vector model and a domain ontology library. Constraint binding relationships are established using a rule inference engine. The rule inference engine dynamically establishes binding relationships by matching the attributes of the task target tree nodes according to the conditional parts of the constraint rules. The conflict resolution protocol employs a multi-round negotiation mechanism. The first round detects the conflict type, the second round selects different resolution strategies based on the conflict type, and the third round executes the resolution operation and verifies the resolution result. The structured task framework is serialized and stored in JSON or XML format for easy parsing and use by other units in the system.

[0078] Optionally, the construction of the scene entity relationship network can incorporate a domain ontology to enhance the semantic expressiveness of entities and relationships. The domain ontology defines the hierarchical relationships and attribute constraints between entity types in the professional scene. The construction of the task goal tree can adopt an iterative refinement approach, first building a coarse-grained task hierarchy and then gradually decomposing it into finer-grained leaf nodes. The management of the constraint rule set can implement version control functionality, recording the modification history of constraint rules and supporting retrospective viewing.

[0079] Optionally, the attribute matching process can incorporate a fuzzy matching algorithm to handle inconsistent attribute requirements. The fuzzy matching algorithm calculates a similarity score between the attribute value and the required value, and sets a threshold to determine whether a match is made. Constraint binding relationships can be dynamically adjusted; when the environment changes during task execution, the system can recalculate the binding relationship between constraint rules and task nodes. Conflict resolution protocols can integrate machine learning components to optimize the selection of resolution strategies by analyzing historical conflict resolution cases.

[0080] It is understandable that hierarchical semantic parsing methods can adapt to occupational scenario descriptions of varying complexity, ensuring the comprehensiveness and accuracy of semantic decomposition through modular processing. The mapping between scenario entity relationship networks, task goal trees, and constraint rule sets creates a unified task representation, providing a structured foundation for subsequent simulation training and evaluation. The conflict resolution protocol design ensures the system's robustness in handling inherent uncertainties and contradictions in real-world scenarios. It is also understandable that the implementation of the task parsing unit fully considers the characteristics of occupational scenario simulation training, achieving the transformation from textual description to a computable task model by combining natural language processing technology and domain knowledge. The structured task framework not only includes the logical structure of the task itself but also incorporates information about the execution environment and constraints.

[0081] Example 2: See Figure 3The dynamic behavior modeling unit constructs a dynamic behavior rule base based on historical training data. This process involves extracting typical behavior pattern fragments from historical training logs, performing cluster analysis on these fragments to form behavior pattern templates, performing condition-action pair parsing on each template to identify trigger condition sets and corresponding action sets, using state machine representation to formally describe the trigger condition sets and corresponding action sets to generate standardized behavior rules, and indexing and arranging these standardized behavior rules according to training scenario types to construct the dynamic behavior rule base. Historical training logs are derived from complete records of previous college student career scenario simulation training, including timestamps, student identities, interaction events, and behavior results. Typical behavior pattern fragments are extracted from historical training logs using a sliding window technique, with the window size dynamically adjusted based on the behavior duration to ensure the fragment contains a complete behavior sequence. Cluster analysis uses an unsupervised learning algorithm to group similar behavior patterns. Behavior pattern templates are stored as prototype vectors, with each template representing a common behavior such as conflict resolution or resource negotiation. Condition-action pair parsing uses natural language processing tools to extract key elements from the templates. Trigger condition sets include environmental variables or event thresholds, and corresponding action sets include student response operations. The state machine representation uses a finite state machine model to describe behavioral rules. State nodes represent behavioral stages, and transition edges represent state changes after a condition is triggered. The indexing is based on a classification system of training scenario types, such as business negotiations or project management. The dynamic behavioral rule base is ultimately implemented using relational database tables, supporting efficient querying and updating.

[0082] In practice, typical behavioral pattern fragments are extracted from historical training logs. These logs are stored in a structured format and include fields such as session identifier, participant role, action type, timestamp, and context parameters. Extraction of typical behavioral pattern fragments is achieved by defining behavioral boundary conditions. These conditions are based on event intervals or semantic integrity; for example, a fragment boundary is considered to be reached when the time difference between consecutive actions exceeds a threshold or when the behavioral objective changes. The extraction process uses a sequence segmentation algorithm. The algorithm scans the historical training logs and identifies consecutive record segments that meet the boundary conditions. Each typical behavioral pattern fragment includes a start time, end time, a list of participating students, and a sequence of behavioral events. Each event in the sequence is labeled with its event type and associated data. The typical behavioral pattern fragments are then converted into feature vector representations. These feature vectors include numerical features such as event frequency, duration, and number of participants, as well as categorical features such as behavioral theme or interaction pattern, preparing for subsequent clustering analysis.

[0083] In practice, typical behavioral pattern fragments are clustered to form behavioral pattern templates. The clustering analysis uses the K-means algorithm, which takes the multidimensional feature vectors of typical behavioral pattern fragments as input, calculates the similarity between vectors, and assigns the fragments to multiple clusters based on the similarity. The objective function of the K-means algorithm is to minimize the sum of squared errors within each cluster, expressed by the formula:

[0084]

[0085] Where: symbol Represents the sum of squared errors within the cluster, with the sign... Indicates the preset number of clusters, symbol The feature vector representing a typical behavioral pattern segment, symbol Represents the i-th cluster, symbol Let represent the centroid vector of the i-th cluster.

[0086] In practice, each behavior pattern template undergoes condition-action pair parsing to identify the trigger condition set and corresponding action set. This parsing is based on feature analysis and contextual reasoning of the behavior pattern template. The trigger condition set is derived from the initial event or environmental state of the behavior pattern template. For example, when the behavior pattern template involves task allocation, the trigger condition set might include task readiness status or resource availability conditions. The elements of the trigger condition set are represented in logical expression form, such as "task status = pending allocation" and "resource quantity > 0". The corresponding action set is extracted from the core behavior sequence of the behavior pattern template. This set includes operations that students might perform, such as sending messages, adjusting plans, or requesting assistance. Each action is labeled with its action type, parameters, and expected effect. The parsing process uses a rule engine to match template features with a predefined condition-action pattern library, ensuring consistency in the causal relationship between the trigger condition set and the corresponding action set. The trigger condition set and corresponding action set are ultimately stored as key-value pairs for easy subsequent formal description.

[0087] In practical implementation, a state machine representation is used to formally describe the trigger condition set and corresponding action set to generate standardized behavioral rules. The state machine representation is implemented using a finite state machine model, which consists of a set of states, transition conditions, and action functions. The state set includes an initial state, intermediate states, and a final state. Each state corresponds to a stage of the behavioral pattern template, such as a "waiting for trigger" state or an "executing action" state. Transition conditions are derived from the trigger condition set and are described by Boolean expressions as the prerequisites for state changes, such as triggering a transition when an environmental variable meets a specific value. Action functions are mapped to the corresponding action set, defining the sequence of operations executed during state transitions. Standardized behavioral rules are encoded in XML or JSON format, containing information such as rule identifiers, applicable scenarios, state machine diagrams, and rule priorities. The formal description process verifies the completeness and unambiguity of the rules through a state machine compiler, ensuring that the standardized behavioral rules can be correctly interpreted by the execution engine during simulation training.

[0088] In implementation, a dynamic behavior rule library is constructed by indexing and arranging standardized behavior rules according to training scenario types. The training scenario types are based on a professional scenario classification system, such as team collaboration scenarios or crisis management scenarios. The classification system adopts a hierarchical structure, with the top level representing the major scenario categories and the lower levels representing specific scenario variations. The indexing and arrangement process groups standardized behavior rules by scenario type, creating index entries for each group. Each index entry includes the scenario type code, a rule list, and matching weights. The dynamic behavior rule library is implemented in the form of database tables. The main table stores rule metadata, and the detail tables store rule content. The index structure uses B-trees or hash indexes to support fast retrieval. During construction, the rule library also integrates a version management mechanism, recording the history of rule additions, modifications, or deletions for easy maintenance and updates. The dynamic behavior rule library is ultimately deployed in the system's rule engine, providing real-time behavior prediction and intervention support for the dynamic behavior modeling unit.

[0089] In some embodiments, the extraction of typical behavioral pattern fragments can incorporate semantic analysis techniques to enhance the accuracy of fragment boundary recognition. Semantic analysis analyzes text descriptions and event contexts in historical training logs, using topic models to identify coherent changes in behavioral topics, thereby determining more reasonable fragment segmentation points. Cluster analysis can employ hierarchical clustering methods instead of the K-means algorithm. Hierarchical clustering merges similar fragments from the bottom up, forming a tree-like cluster structure, facilitating the generation of multi-granularity behavioral pattern templates. Condition-action pair parsing can integrate machine learning classifiers. The classifier learns the mapping relationship between conditions and actions from a large amount of labeled data, improving the automation and coverage of the parsing process.

[0090] In some embodiments, the state machine representation can be extended to a temporal logic state machine. The temporal logic state machine introduces time constraints and concurrent states, enabling the description of more complex temporal relationships of behavior. Index orchestration can implement dynamic weight adjustment, dynamically updating the priority of rules in the index based on rule usage frequency and effect feedback, thus optimizing rule matching efficiency. The dynamic behavior rule base can support an online learning mechanism, which updates the rule content in real time based on newly generated training data, giving the rule base adaptive evolution capabilities.

[0091] Optionally, feature extraction of typical behavioral pattern fragments can incorporate domain-specific features. These features are designed based on professional scenario knowledge, such as communication effectiveness indicators or decision quality scores, enriching the expressive power of feature vectors. The distance metric for clustering analysis can employ Mahalanobis distance to handle feature correlation. Mahalanobis distance considers the feature covariance matrix, improving the robustness of clustering to outliers. The rule engine for condition-action pair parsing can be configured with a fallback strategy, automatically invoking default rules or requesting manual intervention when parsing fails, ensuring system reliability. Optionally, the formal description of the state machine representation can be checked using formal verification tools. These tools verify the deadlock-free nature and reachability of the state machine, ensuring the logical correctness of standardized behavioral rules. Indexed and orchestrated database tables can be partitioned, by scenario type or time range, improving query performance for large-scale rule data. The access interface for the dynamic behavioral rule library can be designed as a RESTful API, facilitating integration with other system units and distributed deployment.

[0092] Understandably, the method of building a dynamic behavior rule base based on historical training data utilizes data-driven approaches to capture behavioral patterns. Cluster analysis summarizes scattered behavioral patterns into reusable templates, improving the efficiency of behavior modeling. State machine representation provides a rigorous formal foundation, making behavioral rules computable and predictable, suitable for simulating complex professional scenarios. Index orchestration optimizes rule organization according to scenario type, supports rapid context matching, and enhances the system's adaptability in changing environments. The construction process of the dynamic behavior rule base emphasizes the value of historical data and the manageability of rules, achieving the transformation from raw logs to structured knowledge through a modular process. Standardized behavior rules ensure the consistency and scalability of the behavior model, providing a reliable behavioral benchmark for simulation training.

[0093] Example 3: The multimodal interaction management unit captures real-time voice interaction data, action sequence data, and gaze focus data between students during simulated training. It then timestamps and fuses these data to generate a multimodal interaction map. Voice interaction data is acquired using a high-fidelity microphone array, action sequence data is recorded using an inertial measurement unit and an optical motion capture system, and gaze focus data is obtained using a head-mounted eye tracker. Voice content features, intonation features, and speech rate features are extracted from the voice interaction data. Voice content features are converted to text using automatic speech recognition technology, and keywords and semantic vectors are extracted. Intonation features are calculated by analyzing the fundamental frequency profile and intensity changes. Speech rate features are determined based on the number of syllables recognized per unit time. Joint motion trajectory features, motion amplitude features, and motion frequency features are extracted from the motion sequence data. Joint motion trajectory features are calculated from the three-dimensional coordinate sequence of skeletal joints, calculating the curvature and direction of the motion path. Motion amplitude features are quantified using the maximum value and variance of joint displacement. Motion frequency features are analyzed using Fourier transform to determine the repetition period of the motion. This study extracts gaze target features, gaze duration features, and saccade path features from gaze focus data. Gaze target features identify specific objects or areas focused on by students in the virtual environment. Gaze duration features statistically analyze the distribution of continuous gaze time, and saccade path features quantify changes in the angle and speed of gaze movement. A unified timeline is established to align multi-source features according to time windows, with the window size dynamically adjusted based on the rhythm of the simulated training scenario. An attention mechanism is used to weightedly fuse the aligned multi-source features to generate a multimodal interaction graph. The attention mechanism calculates the correlation between each feature dimension and the current task and assigns weights. The fused feature vectors are organized into a graph structure according to the time series.

[0094] In the specific implementation, speech content features, intonation features, and speech rate features are extracted from the voice interaction data. The speech content feature extraction process first uses a pre-trained speech recognition model to convert the audio stream into a text sequence. Then, noun entities and action verbs are extracted from the text as keywords. Simultaneously, a semantic encoder maps the entire text into a fixed-dimensional semantic vector. Intonation feature analysis targets the fundamental frequency trajectory of the audio signal. The fundamental frequency trajectory is calculated using an autocorrelation algorithm, and the standard deviation and slope of the fundamental frequency are extracted as indicators of intonation variation. Speech rate feature calculation is based on the effective speech segments segmented by speech activity detection. The number of syllables per second is counted, and the moving average and coefficient of variation of the speech rate are calculated. Feature extraction from the voice interaction data is segmented in 50-millisecond frames to ensure that the temporal resolution of the features meets real-time requirements.

[0095] In practice, joint motion trajectory features, motion amplitude features, and motion frequency features are extracted from the motion sequence data. The motion sequence data comes from sensor nodes distributed across the student's major joints, with each node providing triaxial acceleration and triaxial angular velocity data. Joint motion trajectory features are reconstructed using a sensor data fusion algorithm to reconstruct the three-dimensional motion path of the joints. The curvature feature of the path is calculated using the Flyner formula to determine the degree of local bending, and the direction feature is represented by quaternions to indicate the Euler angles of joint rotation. Motion amplitude features are extracted from the displacement distance of the joints relative to the origin of the torso, and the maximum displacement value and displacement variance within the motion cycle are calculated as amplitude indicators. Motion frequency features are identified by performing a Fast Fourier Transform on the acceleration data to identify the dominant frequency components in the amplitude spectrum. The motion frequency feature is calculated as the weighted average of the first three dominant frequency components. Feature extraction from the motion sequence data is performed using a sliding calculation with a 100-millisecond time window to ensure complete capture of the motion dynamics.

[0096] In practice, gaze focus data is used to extract gaze target features, gaze duration features, and saccade path features. The gaze focus data comes from pupil position and corneal reflex data collected by an eye tracker. The gaze target features determine the student's gaze point coordinates in the virtual environment using a gaze projection algorithm and match them with a scene object database to identify the specific object being gazed at. The gaze duration features are based on a gaze point stability threshold; periods where the gaze movement speed is below the threshold in consecutive sampling points are merged into gaze intervals, and the duration of each gaze interval is calculated. The saccade path features analyze the rapid movement trajectory between gaze points, extracting the average angle, peak velocity, and arc length of the saccade motion as path features. Feature extraction from the gaze focus data is synchronized with the display device's refresh rate, with real-time calculations performed at 16.7 millisecond intervals.

[0097] In implementation, a unified timeline is established to align speech content features, intonation features, speech rate features, joint movement trajectory features, movement amplitude features, movement frequency features, gaze target features, gaze duration features, and saccade path features according to time windows. The unified timeline is based on the system's high-precision clock, and each data sampling point is timestamped at the microsecond level. The time window size is dynamically adjusted according to the simulation training phase, using a short window of 200 milliseconds during the rapid interaction phase and a long window of 500 milliseconds during the policy decision phase. The alignment process uses an interpolation algorithm to process data streams with different sampling rates, ensuring that all features have corresponding values ​​at the same time point. The feature data within the time window is organized in a multi-dimensional matrix form, with rows corresponding to time points and columns corresponding to feature dimensions.

[0098] In practice, an attention mechanism is used to perform weighted fusion of the aligned multi-source features. The attention mechanism first calculates the relevance score of each feature dimension to the core task of the current training phase. The relevance score is calculated through query-key-value matching.

[0099]

[0100] Where: symbol Represents the attention score matrix, with the symbol... A query vector representing the current task status, symbol Represents a key vector matrix for all feature dimensions, with the symbol... The dimension of the feature vector is represented by the symbol. This represents the matrix transpose operation. Initial attention weights are assigned based on relevance scores and normalized using a softmax function. Contextual information from a dynamic behavior rule library is introduced to modify the initial attention weights. This contextual modification adjusts the weight distribution through a gating mechanism that considers the importance of the current behavior rule and historical attention patterns. The modified attention weights are then used to perform a weighted summation on the corresponding features to generate a fused feature vector. The fused feature vector is organized into graph nodes according to time series, and edges are connected based on the temporal dependencies between features to construct a multimodal interaction graph. The multimodal interaction graph adopts a directed graph structure, where node attributes contain the fused feature vector, and edge weights represent feature transition probabilities.

[0101] In some embodiments, speech content feature extraction can integrate a domain-specific dictionary to enhance the recognition accuracy of professional terms. This domain-specific dictionary contains professional vocabulary and expressions related to specific professional scenarios. Joint motion trajectory features from action sequence data can be processed using a dynamic time warping algorithm to address the differences in the rhythm of different students' movements. The dynamic time warping algorithm optimizes the temporal alignment of trajectory features. Gaze path features from gaze focus data can be combined with scene depth information to calculate the actual distance of gaze movement. Scene depth information is obtained from a 3D model of the virtual environment. The time window alignment process can employ a dynamic programming algorithm to optimize the interpolation path, minimizing the temporal error between different signal sources. Context correction in the attention mechanism can introduce a multi-head attention structure. This structure captures feature correlations from different subspaces, improving the expressive power of fused features. The construction of a multimodal interaction graph can implement an incremental update mechanism. This incremental update mechanism only processes the changed parts of the graph structure, improving the maintenance efficiency of large-scale graphs.

[0102] Optionally, the speech rate features of voice interaction data can be supplemented with the proportion of silent segments as a secondary indicator, reflecting the thought pause patterns in the dialogue. The action amplitude features of action sequence data can be normalized to height proportions to eliminate the influence of individual body size differences. The gaze target features of gaze focus data can integrate target semantic information, which describes the functional significance of the gazed object in the task context. Optionally, the synchronization signal for a unified timeline can originate from a hardware time synchronization module, which ensures clock consistency across multiple devices via GPS or network time protocols. Regularization constraints can be incorporated into the weight adjustment process of the attention mechanism to prevent overfitting, limiting the range of weight changes. The storage of the multimodal interaction graph can employ graph database technology, which supports complex graph query and traversal operations.

[0103] It is understandable that feature extraction from multimodal data covers three key dimensions: communication, behavior, and visual attention, providing a data foundation for a comprehensive understanding of interaction dynamics. Timestamp alignment technology solves the integration challenge of multi-source asynchronous data, ensuring temporal consistency in feature analysis. The attention mechanism-based feature fusion method adapts to the task priorities of different training stages, achieving focused multimodal information integration. It is also understandable that the construction of a multimodal interaction graph transforms temporal features into a spatial topological structure, facilitating subsequent state evolution and decision analysis. The entire implementation process emphasizes a balance between real-time processing efficiency and feature quality, meeting the dual requirements of low latency and high accuracy for simulation training systems. The technical solution of the multimodal interaction management unit fully leverages the complementary advantages of multi-source data, providing a rich foundation for interaction analysis in professional scenario simulation.

[0104] Example 4: The state evolution tracking unit identifies individual students' decision nodes based on the multimodal interaction graph and tracks the team state change path triggered by these decision nodes, recording the state evolution sequence. At each decision node, the decision-making subject and decision content are marked. The decision-making subject is determined by the binding relationship between behavioral patterns and student identities in the multimodal interaction graph, while the decision content is parsed from semantic analysis of voice interaction data and intent recognition of action sequence data. The unit simulates the execution of the decision content to deduce the reaction behaviors of other students in subsequent time steps. This deduction process is based on behavioral rules and state transition logic in a dynamic behavior rule base. The unit verifies the actual occurrence of reaction behaviors based on the multimodal interaction graph and records the deviation between actual and expected behaviors. The deviation record includes a quantitative description of the deviation amount and type. A state evolution sequence is formed by recording the causal chain between decision nodes, actual reaction behaviors, and deviations in a graph structure. The graph structure uses a directed acyclic graph, where nodes represent state events and edges represent causal relationships.

[0105] In practical implementation, decision-making entities and decision content are marked at decision nodes. Decision node identification is achieved by analyzing feature mutation points in the multimodal interaction graph. Feature mutation points include the appearance of key command words in voice interaction data, significant posture changes in action sequence data, or gaze target switching in gaze focus data. Decision-making entity marking associates decision-making behavior with specific students through a student identity recognition module based on biometrics or role assignment information. Decision content marking uses natural language processing technology to analyze the semantic content in voice interaction data and combines it with the classification results of action sequence data to determine the operational intent of the decision. Referring to Table 1, the marking information is stored as a decision node attribute table, which includes fields such as node number, timestamp, decision-making entity identifier, and decision content description.

[0106] Table 1: Decision Node Attribute Table.

[0107]

[0108] In practice, the simulation executes the decision content and extrapolates the reaction behaviors of other students in subsequent time steps. The simulation executes condition-action rules based on a dynamic behavior rule base, and the rule engine generates a sequence of reaction behaviors based on the decision content and the current environmental state. The extrapolation process employs discrete event simulation technology, modeling each reaction behavior as an event with a time delay. The event delay is determined based on the behavior type and historical data statistics. Reaction behaviors include direct response behaviors and indirect chain reaction behaviors. The extrapolation depth is set to three time steps to balance computational complexity and prediction completeness. The output of the reaction behaviors is a sequence of behaviors with probabilities, where each behavior is labeled with its probability of occurrence and scope of influence.

[0109] In practice, the deviation between actual and expected behaviors is recorded based on the multimodal interaction graph to verify the actual occurrence of response behaviors. The verification process involves temporal matching between the observed behavior sequences and the inferred response behavior sequences in the multimodal interaction graph. The deviation is calculated by comparing the differences between actual and expected behaviors on key features. The difference measurement uses a multidimensional feature distance formula:

[0110]

[0111] Where: symbol Indicates the deviation amount, symbol Indicates the number of feature dimensions, symbol The weight coefficient of the j-th feature is represented by the symbol. The j-th feature value representing the actual behavior, symbol The j-th feature value represents the expected behavior. Deviation types are classified based on the pattern of feature differences; the main deviation types include time-series deviation, intensity deviation, and direction deviation. Deviation records are stored in a structured deviation log format, which includes the deviation number, associated decision node, deviation value, and deviation type code.

[0112] In implementation, a graph structure is used to record the causal chain between decision nodes, actual reaction behaviors, and deviations, forming a state evolution sequence. The graph structure employs a node-edge model, with decision nodes serving as the root nodes and actual reaction behaviors as child nodes connected to their respective root nodes. Behavior type and timestamp are used as node attributes; behavior types are selected from a predefined behavior classification system, and timestamps are accurate to milliseconds. Deviations are displayed as edge attributes on the edges connecting root nodes and child nodes, with the annotations including the specific numerical values ​​of the deviation amount and type. When multiple decision nodes have temporal or logical connections, cross-node connections are established between corresponding root nodes, representing the propagation path of state changes. The state evolution sequence is stored using graph database technology, supporting efficient path lookup and backtracking analysis.

[0113] In some embodiments, the identification of decision nodes can incorporate a multimodal feature fusion algorithm to improve detection accuracy. This algorithm simultaneously analyzes the collaborative patterns of speech, action, and gaze features, reducing false positives. The inference of reaction behavior can be integrated with a machine learning prediction model. This model learns behavioral response patterns from historical interaction data, supplementing the coverage of the rule base. The deviation verification process can implement a real-time alarm mechanism, automatically triggering system intervention when the deviation exceeds a threshold.

[0114] In some embodiments, the construction of causal relationship chains can employ probabilistic graphical models to enhance expressive power, where conditional probabilities are added to edges to represent the strength of causal relationships. The analysis of state evolution sequences can incorporate time-series pattern mining algorithms, which identify periodic or trend-based patterns in state changes. Visualization of the graph structure can achieve dynamic display functionality, presenting the state evolution process in chronological order. Optionally, the labeling of decision-making entities can support role-group level abstraction; when multiple students collaborate on decision-making, the decision-making entities are labeled as role groups rather than individuals. The deduction of reaction behaviors can be configured with multiple scenario modes, each corresponding to different team collaboration styles or emergency strategies. Deviation records can include environmental influencing factors, describing the contribution of changes in external conditions to behavioral deviations.

[0115] Optionally, the storage of causal relationship chains can adopt an incremental update strategy, which only processes newly added decision nodes and related edges, improving the processing efficiency of large-scale evolutionary sequences. The query interface for state evolution sequences can support multi-condition filtering, including dimensions such as time range, decision type, or deviation level. It can be understood that precise labeling of decision nodes provides a reliable starting point for state evolution analysis, ensuring the accuracy of the tracing path. The behavior inference and verification mechanism creates a comparison framework between expectations and reality, providing quantitative basis for team behavior evaluation. The graph-structured state evolution sequence intuitively displays the causal relationship of state changes, supporting in-depth interaction pattern analysis. The implementation of the state evolution tracking unit achieves refined tracking of team dynamics through multimodal data fusion and rule-based reasoning technology. The deviation recording mechanism captures detailed characteristics of behavioral differences, providing data support for personalized feedback. The entire solution emphasizes the integrity of temporal relationships and the interpretability of causal logic, meeting the analytical needs of professional training scenarios.

[0116] See Figure 4 In the deviation quantification statistics of the state evolution tracking unit, the distribution of the average deviation of each decision node intuitively presents the degree of difference between the actual behavior and the expected behavior caused by different decisions. Specifically, the figure uses the decision node number as the horizontal axis and the average deviation as the vertical axis, and displays the average deviation of six decision nodes from DN001 to DN006 in the form of a bar chart, with a deviation threshold of 0.2 marked by a dashed line. From the data distribution, it can be seen that: DN004 has the highest average deviation (0.34), which significantly exceeds the deviation threshold; the average deviations of DN002 (0.27) and DN006 (0.21) are also higher than the threshold, belonging to high-risk deviation nodes; while the average deviations of DN001 (0.11), DN003 (0.07), and DN005 (0.17) are lower than the threshold, and the behavioral consistency is relatively good. This statistical result is based on behavioral feature matching of multimodal interaction graph and calculation of multidimensional feature distance formula. The quantification of deviation integrates the weight differences of features such as voice and action. Its distribution can directly reflect the degree to which the team behavior deviates from the expectation at different decision nodes. It provides core quantitative basis for the subsequent adjustment of intervention priority of strategy adaptation unit and the calculation of performance consistency index of comprehensive evaluation generation unit.

[0117] Example 5: The strategy adaptation unit matches candidate strategies from a dynamic behavior rule base based on state evolution sequences and adjusts the weights of these candidate strategies according to real-time training environment parameters, outputting an adapted intervention strategy. These parameters include task progress metrics and team collaboration metrics. The unit monitors task progress and team collaboration metrics in the training scenario in real time. Task progress metrics are quantified by task completion percentage and milestone achievement time, while team collaboration metrics are measured by communication frequency and conflict resolution rate. The task progress metrics are compared with preset milestones to calculate progress deviation, reflecting the difference between actual and planned progress. Team collaboration metrics are compared with an ideal collaboration model to calculate collaboration efficiency values, representing how close the current team collaboration level is to the ideal state. The activation threshold and execution priority of candidate strategies are dynamically adjusted based on progress deviation and collaboration efficiency values. The activation threshold determines the strictness of the triggering conditions, and the execution priority controls the execution order of the strategies. Based on the adjusted activation threshold and execution priority, an adapted intervention strategy is selected from the candidate strategies. This adapted intervention strategy is the most suitable behavioral guidance scheme for the current training state. The comprehensive evaluation unit calculates consistency indicators of individual and team performance across multiple dimensions to generate a structured evaluation report, defining three evaluation dimensions: task execution, communication and coordination, and emergency response. Under the task execution dimension, a matching coefficient is calculated between individual task completion and team task synchronization; this coefficient measures the coordination between individual contribution and team progress. Under the communication and coordination dimension, a correlation index is analyzed between individual communication initiative and team information flow efficiency; this index reflects the promoting effect of individual communication behavior on team information sharing. Under the emergency response dimension, the coordination ratio between individual decision-making speed and team crisis recovery speed is assessed; this ratio reflects the matching degree between individual emergency response and team recovery capability. The matching coefficient, correlation index, and coordination ratio are normalized and weighted to generate a performance consistency index, which comprehensively reflects the degree of coordination between individual and team performance.

[0118] In practical implementation, real-time monitoring of task progress and team collaboration metrics in the training scenario is performed. Task progress metrics are obtained from the task management subsystem, including specific indicators such as subtask completion status, resource consumption rate, and time utilization rate. Team collaboration metrics are extracted from the multimodal interaction graph, including interaction features such as dialogue turn distribution, opinion adoption rate, and response latency. Data is collected at a rate of seconds, and the metric data undergoes moving average filtering to eliminate the impact of instantaneous fluctuations. Real-time training environment parameters are stored in a time-series database, supporting rapid querying and historical trend analysis. The monitoring module also includes anomaly detection functionality, triggering a data verification process when metric values ​​exceed normal ranges. Team collaboration metrics are compared with an ideal collaboration model to calculate collaboration efficiency values. The ideal collaboration model is built based on historical high-performing training cases and includes optimal communication patterns and behavioral norms. Collaboration efficiency values ​​are calculated by comparing the matching degree between actual collaboration metrics and ideal values, using a weighted similarity algorithm to measure the closeness of the current collaboration state to the ideal model. The calculation of collaboration efficiency values ​​comprehensively considers multiple aspects such as communication frequency, information accuracy, and decision quality, assigning corresponding weight coefficients to each aspect. The collaboration efficiency value is standardized to a range of 0 to 1, with values ​​closer to 1 indicating higher collaboration efficiency. The collaboration efficiency value is updated every five minutes, and the trend of change is recorded for subsequent analysis.

[0119] In implementation, the activation threshold and execution priority of candidate strategies are dynamically adjusted based on progress deviation and collaboration efficiency. The activation threshold adjustment employs a reverse proportional mechanism: when progress deviation increases, the activation threshold is appropriately lowered; when collaboration efficiency decreases, the threshold sensitivity is adjusted accordingly. Execution priority adjustment is based on a comprehensive scoring algorithm, which considers factors such as the strategy's match with the current problem, the strategy's historical success rate, and the strategy's implementation cost. The dynamic adjustment process is executed every two minutes to ensure that strategy parameters respond promptly to changes in the training state. The adjusted activation threshold and execution priority are updated to the strategy database for use by the strategy selection module. Based on the adjusted activation threshold and execution priority, suitable intervention strategies are selected from the candidate strategies. The selection process first filters out candidate strategies that do not meet the triggering conditions based on the activation threshold, and then sorts the remaining strategies according to their execution priority. The three highest-priority strategies are selected to form a strategy set, which includes the main strategy and backup strategies. The specific content of the suitable intervention strategy is extracted from the dynamic behavior rule base, including strategy description, implementation steps, and expected effects. The suitable intervention strategy is pushed to the training instructor through the system interface and simultaneously transmitted to other relevant units in a standardized format.

[0120] In practice, performance consistency indicators for individuals and teams across multiple dimensions are calculated. The matching coefficient for task execution is obtained by calculating the Pearson correlation coefficient between individual task completion and team task synchronization. The correlation index for communication and coordination uses regression analysis to determine the contribution of individual communication initiative to team information flow efficiency. The coordination ratio for emergency response is calculated as the ratio of individual decision-making speed to team crisis recovery speed, and logarithmic transformation is used to ensure symmetrical distribution. Normalization employs a min-max standardization method to transform each dimension's indicators to the 0-1 range, and weighted aggregation uses weight coefficients determined by the analytic hierarchy process. The final value of the performance consistency indicator is calculated through weighted summation; a higher value indicates greater coordination between individual and team performance.

[0121] In some embodiments, the monitoring of task progress indicators can introduce predictive metrics, which predict the final completion status based on the current progress trend, enhancing the foresight of progress management. The comparison of team collaboration indicators can employ a dynamic baseline adjustment mechanism, where the ideal value standard is adaptively adjusted according to the characteristics of the training phase. The weight adjustment of candidate strategies can integrate reinforcement learning algorithms, which dynamically optimize and adjust parameters based on the strategy execution effect. The calculation of performance consistency indicators can incorporate a time decay factor, which reduces the influence of early data on current indicators, highlighting the importance of recent performance. The weight coefficients of weighted aggregation can be determined through expert review, which integrates the experience and judgment of experts from multiple domains. The generation of structured evaluation reports can achieve automated template customization, automatically selecting the most suitable report format based on the training type.

[0122] Optionally, the calculation of progress deviation can distinguish between positive and negative deviations. Positive deviation indicates that the progress is ahead of schedule, while negative deviation indicates that the progress is behind schedule, and different processing strategies are adopted for each. The calculation of collaboration efficiency value can introduce a context adjustment factor, which considers the impact of changes in training difficulty on collaboration requirements. The dimensional weights of the performance consistency index can be dynamically adjusted, and the importance of each dimension can be redistributed according to changes in training focus. Optionally, the push of adaptive intervention strategies can be in a multimodal form, including text prompts, visual signals, and voice reminders. The structured evaluation report can generate multi-granular versions, including detailed versions, summary versions, and visualization versions, to meet the needs of different use cases.

[0123] See Figure 5In the parameter adjustment and effect evaluation of the strategy adaptation unit, the dynamic changes and correlations of activation threshold, execution priority (normalized), strategy matching degree (%), and historical success rate (%) are intuitively presented under each training stage (task parsing, dynamic modeling, multimodal interaction, state evolution, strategy adaptation and evaluation). Specifically, the activation threshold (blue curve) has an initial value of 0.75 in the task parsing stage, decreases slightly in the dynamic modeling stage, and then gradually rises to 0.85 in the strategy adaptation and evaluation stage; the execution priority (orange curve) shows a fluctuating characteristic of "first decreasing and then rising", dropping to 0.60 in the dynamic modeling stage and rising back to 1.00 in the strategy adaptation and evaluation stage; the trend of strategy matching degree (green curve) and historical success rate (red curve) is highly correlated. Both are at a low point in the dynamic modeling stage (78% and 70%, respectively), and rise to a peak in the multimodal interaction stage (95% and 90%). Although they subsequently decline, they rise again to near the peak level in the strategy adaptation and evaluation stage. This multi-indicator linkage reflects the dynamic adjustment effect of real-time training environment parameters (task progress, collaboration efficiency) on the activation threshold and execution priority of candidate strategies, as well as the positive correlation between strategy parameter adjustment and strategy matching degree and historical success rate.

[0124] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0125] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A university student professional scene simulation training evaluation system, characterized in that, The system comprises: a task analysis unit configured to receive an initial professional scene description, perform semantic decomposition on the initial professional scene description, extract scene core task elements and role constraint conditions, and generate a structured task framework; a dynamic behavior modeling unit configured to import a preset student role configuration according to the structured task framework, assign initial behavior parameters to each student role configuration, and construct a dynamic behavior rule library based on historical training data; a multi-modal interaction management unit configured to capture voice interaction data, action sequence data, and visual focus data between students in real time during simulation training, timestamp align and feature fuse the voice interaction data, action sequence data, and visual focus data, and generate a multi-modal interaction graph; a state evolution tracking unit configured to identify decision nodes of individual students according to the multi-modal interaction graph, track team state change paths triggered by the decision nodes, and record state evolution sequences; a strategy adaptation unit configured to match candidate strategies in the dynamic behavior rule library based on the state evolution sequences, weight adjust the candidate strategies according to real-time training environment parameters, and output an adapted intervention strategy; a comprehensive evaluation generation unit configured to integrate the state evolution sequences and the adapted intervention strategy, calculate performance consistency indicators of individuals and teams in multiple dimensions, and generate a structured evaluation report.

2. The university student professional scene simulation training evaluation system according to claim 1, wherein, The semantic decomposition on the initial professional scene description comprises: adopting a hierarchical semantic analysis method to split the initial professional scene description into a scene background section, a task target section, and a constraint condition section; performing entity recognition and relationship extraction on the scene background section to establish a scene entity relationship network; performing action verb analysis and logical dependency analysis on the task target section to construct a task target tree; performing rule clause classification and priority assignment on the constraint condition section to form a constraint rule set; associating and mapping the scene entity relationship network, the task target tree, and the constraint rule set to generate the structured task framework.

3. The university student professional scene simulation training evaluation system according to claim 1, wherein, The construction of the dynamic behavior rule library based on historical training data comprises: extracting typical behavior mode segments from historical training logs, performing cluster analysis on the typical behavior mode segments to form behavior mode templates; performing condition-action pair analysis on each behavior mode template to identify a trigger condition set and a corresponding action set; adopting a state machine representation method to formally describe the trigger condition set and the corresponding action set to generate standardized behavior rules; indexing and arranging the standardized behavior rules according to training scene types to construct the dynamic behavior rule library.

4. The university student professional scene simulation training evaluation system according to claim 1, wherein, The timestamp alignment and feature fusion of the voice interaction data, action sequence data, and visual focus data comprise: extracting voice content features, tone features, and speech rate features for the voice interaction data; extracting joint motion trajectory features, action amplitude features, and action frequency features for the action sequence data; extracting fixation target features, fixation duration features, and saccade path features for the visual focus data; align the voice content features, the intonation features, the speech rate features, the articulatory trajectory features, the action amplitude features, the action frequency features, the gaze target features, the gaze duration features, and the saccade path features according to a time window; perform weighted fusion on the aligned multi-source features by using an attention mechanism to generate the multi-modal interaction graph.

5. The university student professional scene simulation training evaluation system according to claim 1, wherein, The tracking of the team state change path triggered by the decision node includes: labeling a decision subject and decision content at the decision node; simulating execution of the decision content to deduce reaction behaviors of other students in a subsequent time step; verifying actual occurrence of the reaction behaviors according to the multi-modal interaction graph, and recording deviations between actual behaviors and expected behaviors; recording a causal relationship chain between the decision node, the actual reaction behaviors, and the deviations in a graph structure to form the state evolution sequence.

6. The university student professional scene simulation training evaluation system according to claim 1, wherein, The weight adjustment of the candidate strategy according to real-time training environment parameters includes: monitoring task progress indicators and team collaboration indicators in a training scene in real time; comparing the task progress indicators with preset milestones to calculate a progress deviation degree; comparing the team collaboration indicators with an ideal collaboration model to calculate a collaboration efficiency value; dynamically adjusting an activation threshold and an execution priority of the candidate strategy according to the progress deviation degree and the collaboration efficiency value; selecting the adaptive intervention strategy from the candidate strategy based on the adjusted activation threshold and execution priority.

7. The university student professional scene simulation training evaluation system according to claim 1, wherein, The calculation of the performance consistency indicators of individuals and teams in multiple dimensions includes: defining a task execution dimension, a communication and coordination dimension, and an emergency response dimension; in the task execution dimension, calculating a matching coefficient of individual task completion degree and team task synchronization degree; in the communication and coordination dimension, analyzing a correlation index of individual communication initiative and team information flow efficiency; in the emergency response dimension, evaluating a coordination ratio of individual decision speed and team crisis recovery speed; normalizing and weightedly aggregating the matching coefficient, the correlation index, and the coordination ratio to generate the performance consistency indicators.

8. The university student professional scene simulation training evaluation system according to claim 2, wherein, The association and mapping of the scene entity relationship network, the task target tree, and the constraint rule set include: identifying key entity nodes in the scene entity relationship network, and performing attribute matching between the key entity nodes and leaf nodes of the task target tree; establishing a constraint binding relationship between the constraint rule set and branch nodes in the task target tree; when attribute matching conflicts or constraint binding conflicts are detected, starting a conflict resolution protocol to reassign entity attributes or adjust constraint priorities; after conflict resolution is completed, generating the structured task framework containing complete association relationships of entities, tasks, and constraints.

9. The university student professional scene simulation training evaluation system according to claim 4, characterized in that, The weighted fusion on the aligned multi-source features by using an attention mechanism includes: calculating a relevance score of each feature dimension with a core task in a current training phase; allocating an initial attention weight according to the relevance score; introducing context information in the dynamic behavior rule library to contextually correct the initial attention weight; The corresponding features are weighted and summed using the corrected attention weight to generate a fusion feature vector; The fusion feature vector is organized as a graph node in time sequence and connected by edges according to the time sequence dependence between features to construct the multi-modal interaction graph.

10. The university student professional scene simulation training evaluation system according to claim 5, wherein, The causal relationship chain between the decision node, the actual reaction behavior and the deviation recorded in the form of a graph structure includes: Each decision node is taken as a root node of a causal graph; The actual reaction behavior is connected as a sub-node to the corresponding root node, and the behavior type and time stamp are marked; The deviation is marked as an edge attribute on the edge connecting the root node and the sub-node, and the marking content includes the deviation amount and the deviation type; When multiple decision nodes have time sequence or logical association, a cross-node connection edge is established between the corresponding root nodes to form the state evolution sequence describing the state propagation path.

Images