A simulation training evaluation method, device, equipment, medium and program product

Abstract

The application discloses a simulation training evaluation method, device, equipment, medium and program product. The baseline value of the multi-modal physiological data of a trainee, the real-time value of the multi-modal physiological data and the behavior response data in the process of performing a simulation training task containing preset event nodes are collected. The real-time value of the multi-modal physiological data of the trainee is time-domain aligned with the event nodes with the event nodes as time stamps, an event-physiological alignment data matrix is established, and the multi-modal physiological characteristic value of the trainee under each event node is calculated. According to the baseline value of the multi-modal physiological data, the multi-modal physiological characteristic value and the behavior response data, the behavior performance score and the psychological stress score of the trainee under each event node are calculated, the failure risk prediction and the proficiency rating are performed. The application combines the behavior response and the psychological state to evaluate the ability of the trainee, and improves the accuracy of the evaluation result.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a simulation training and evaluation method, apparatus, device, medium, and program product. Background Technology

[0002] Currently, in virtual simulation training systems used in military and police combat simulations, high-risk industry operation training, and emergency drills, the evaluation of trainees' training effectiveness mainly relies on two types of indicators: objective behavioral data and subjective human evaluation. Objective behavioral data refers to quantifiable behavioral results recorded in system logs, such as task completion time, accuracy of operation sequences, target hit rate, and interaction parameters with key objects in the virtual environment. Subjective human evaluation refers to instructors or evaluators giving subjective scores based on observed trainees' operational standardization and decision-making rationality.

[0003] However, the existing assessment systems mentioned above usually only focus on the training results and ignore the changes in the trainees' psychological state during the task execution process. Trainees may complete the task by chance under high tension or cognitive overload. Such trainees are very prone to making mistakes in real high-pressure scenarios. Existing technologies are unable to identify this risk, resulting in incomplete and subjective assessment results. Summary of the Invention

[0004] The purpose of this invention is to provide a simulation training assessment method, device, equipment, medium, and program product that can assess the trainee's abilities by combining the trainee's behavioral responses and psychological state, thereby effectively improving the accuracy of the assessment results.

[0005] To achieve the above objectives, embodiments of the present invention provide a simulation training evaluation method, comprising: The baseline values ​​of the multimodal physiological data of the trainees are collected, as well as the real-time values ​​and behavioral response data of the multimodal physiological data during the execution of a simulated training task containing preset event nodes; Using the event node as a timestamp as a reference, the real-time values ​​of the trainee's multimodal physiological data are time-domain aligned with the event node to establish an event-physiological alignment data matrix; Based on the event-physiology alignment data matrix, calculate the multimodal physiological feature values ​​of the trainee at each event node; Based on the baseline values ​​of the multimodal physiological data, the multimodal physiological characteristic values, and the behavioral response data, the trainee's behavioral performance score and psychological stress score at each event node are calculated. Error risk prediction and proficiency rating are performed based on the behavioral performance score and the psychological stress score.

[0006] As an improvement to the above scheme, the multimodal physiological data includes at least data from multiple dimensions such as cardiovascular activity data, skin conductance activity data, and eye activity data; The cardiovascular activity data includes heart rate variability index, which is calculated by monitoring the photoplethysmography (PPI) or electrocardiogram (ECG) data of the trainees. The skin conductance activity data includes skin conductance increment index, which is calculated by monitoring the skin conductance activity of the trainee; The eye activity data includes a pupillary oscillation frequency index, which is calculated by monitoring the pupil diameter data of the trainee.

[0007] As an improvement to the above scheme, the event-physiology aligned data matrix includes each event node, the timestamp of the event node, and the real-time value sequence of multimodal physiological data within the physiological data window corresponding to the event node; the physiological data window corresponding to the event node is a time window of a preset length truncated with the timestamp of the event node as the center.

[0008] As an improvement to the above scheme, the step of calculating the multimodal physiological feature values ​​of the trainee at each event node based on the event-physiological alignment data matrix includes: Based on the event-physiological alignment data matrix, feature extraction is performed on the real-time value sequence of multimodal physiological data within each physiological data window to obtain the multimodal physiological feature values ​​of the trainee at each event node.

[0009] As an improvement to the above scheme, the step of calculating the trainee's behavioral performance score and psychological stress score at each event node based on the baseline value of the multimodal physiological data, the multimodal physiological feature value, and the behavioral response data includes: Based on the behavioral response data and preset standard behavioral rules, the trainee's behavioral performance score is calculated; wherein, the behavioral performance score is used to characterize the trainee's level of behavioral standardization. The trainee's psychological stress score is calculated based on the degree of deviation between the multimodal physiological characteristic values ​​and the baseline values ​​of the multimodal physiological data; wherein the psychological stress score is used to characterize the trainee's current psychological load and stress level.

[0010] As an improvement to the above scheme, the step of predicting error risk and rating proficiency based on the behavioral performance score and the psychological stress score includes: Based on the psychological stress sequence at each event node during the current simulation training task and the trainee's fatigue index, predict the trainee's error probability in subsequent simulation training tasks. When the probability of error exceeds a preset warning threshold, a preset alarm message is pushed. When the behavioral performance score is greater than a preset first scoring threshold and the psychological stress score is greater than a preset second scoring threshold, the step size for adjustment is calculated based on the psychological stress score and the error probability. The trainee's current proficiency rating is downgraded based on the downgrade step size to obtain the final proficiency rating after downgrading.

[0011] This invention also provides a simulation training evaluation device, comprising: The data acquisition module is used to collect baseline values ​​of the trainee's multimodal physiological data, as well as real-time values ​​and behavioral response data of the multimodal physiological data during the execution of a simulated training task containing preset event nodes; The data alignment module is used to align the real-time values ​​of the trainee's multimodal physiological data with the event node in the time domain, based on the event node as the timestamp, and establish an event-physiological alignment data matrix. The feature extraction module is used to calculate the multimodal physiological feature values ​​of the trainee at each event node based on the event-physiological alignment data matrix. The comprehensive scoring module is used to calculate the trainee's behavioral performance score and psychological stress score at each event node based on the baseline value of the multimodal physiological data, the multimodal physiological feature value, and the behavioral response data. The risk prediction module is used to predict error risk and assess proficiency based on the behavioral performance score and the psychological stress score.

[0012] This invention also provides a simulation training and evaluation device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the simulation training and evaluation method as described in any of the above embodiments.

[0013] This invention also provides a computer-readable storage medium, which includes a stored computer program, wherein the computer program, when running, controls the device where the computer-readable storage medium is located to execute the simulation training and evaluation method as described in any of the above embodiments.

[0014] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the simulation training and evaluation method as described in any of the above embodiments.

[0015] Compared with existing technologies, the simulation training evaluation method, apparatus, equipment, medium, and program products disclosed in this invention achieve refined and quantitative evaluation of the trainee's psychological state at specific event stages by aligning multimodal physiological data with virtual training event nodes in the temporal domain. This avoids the limitations of traditional evaluations that only focus on behavioral outcomes, and enables parallel evaluation of the trainee's behavioral performance and psychological state during simulated training, effectively improving the accuracy of evaluation results. By constructing an aligned data matrix, a causal relationship between physiological response and external stimuli is established for each training event, making the evaluation results interpretable. Finally, through error risk prediction and dynamic rating adjustment mechanisms, closed-loop management and pre-warning of training effectiveness are achieved, effectively compensating for the blind spot of high scores but low performance in evaluations. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a simulation training evaluation method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a simulation training and evaluation device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a simulation training and evaluation device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] In the description of this application, it should be understood that the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0019] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0021] See Figure 1 This is a flowchart illustrating a simulation training evaluation method provided in an embodiment of the present invention. The embodiment of the present invention provides a simulation training evaluation method, the method comprising steps S11 to S15: S11. Collect baseline values ​​of the trainee's multimodal physiological data, as well as real-time values ​​and behavioral response data of the multimodal physiological data during the execution of a simulated training task containing preset event nodes; S12. Using the event node as a timestamp as a reference, align the real-time values ​​of the trainee's multimodal physiological data with the event node in the time domain to establish an event-physiological alignment data matrix. S13. Calculate the multimodal physiological feature values ​​of the trainee at each event node based on the event-physiological alignment data matrix. S14. Based on the baseline values ​​of the multimodal physiological data, the multimodal physiological characteristic values, and the behavioral response data, calculate the trainee's behavioral performance score and psychological stress score at each event node; S15. Based on the behavioral performance score and the psychological stress score, predict the risk of error and rate the proficiency.

[0022] Preferably, the baseline values ​​for collecting the multimodal physiological data of the trainees are specifically: The average value of the multimodal physiological data of the trainees in their resting state before the start of the simulated training task was collected as the baseline value of the multimodal physiological data.

[0023] In this embodiment of the invention, the trainee wears a wearable device that integrates multimodal physiological sensors, such as a smart wristband sensor, a chest strap sensor, or an MR / VR headset. Before the formal start of the training task, the trainee enters a resting state for 2-3 minutes. The system collects the trainee's multimodal physiological data during this period and calculates the average value as the trainee's individualized physiological baseline value.

[0024] Subsequently, the trainee enters a simulated training task containing preset event nodes to perform operations. For example, preset event nodes in the VR scene include "encountering enemy attack" (10 seconds), "teammate injured and calling for help" (25 seconds), and "hostage rescue" (40 seconds). Throughout the task execution, the system continuously collects real-time values ​​of the trainee's multimodal physiological data and simultaneously records the trainee's behavioral response data. For example, the behavioral response data includes, but is not limited to, target hit rate (e.g., shooting accuracy), reaction time, task completion time, operation sequence, and interaction parameters with key objects in the virtual environment (e.g., distance or orientation to key targets).

[0025] The system uses the timestamp of each event node as a benchmark to align the acquisition timeline of physiological signals with the event log of the VR scene at the millisecond level. Taking the "Encountering Enemy Attack" event node with a timestamp of 10:05:01.200 as an example, the system uses this timestamp as a benchmark and binds and stores the real-time value sequence of multimodal physiological data before and after this timestamp with the event. The same operation is performed on each event node in turn to generate an "event-physiology" aligned data matrix containing all event nodes and their corresponding real-time values ​​of multimodal physiological data. Then, in the event-physiology aligned data matrix, representative feature values ​​are extracted from the real-time value sequence of multimodal physiological data corresponding to each event node to obtain multimodal physiological feature values.

[0026] For each event node, corresponding to multimodal physiological data and behavioral response data, two scores are calculated: a behavioral performance score and a psychological stress score. The behavioral performance score characterizes the trainee's level of behavioral standardization; generally, a higher score indicates a more standard and excellent behavioral response. The psychological stress score characterizes the trainee's current psychological load and stress level; generally, a higher score indicates greater psychological stress and a higher degree of stress.

[0027] Based on the time series of psychological stress scores at each event node in this simulated training task, the changing trend and decline rate of the stress scores are analyzed, and the results are input into the prediction model to calculate the trainee's error probability in subsequent simulated tasks. Simultaneously, if a trainee exhibits a high behavioral performance score but also a high psychological stress score, the system adjusts the trainee's proficiency rating based on the degree of deviation and error probability.

[0028] By employing the technical means of this invention, and through temporal alignment of multimodal physiological data with virtual training event nodes, a refined and quantitative assessment of the trainee's psychological state at specific event stages is achieved. This avoids the limitations of traditional assessments that only focus on behavioral outcomes, enabling parallel assessment of the trainee's behavioral performance and psychological state during simulated training, effectively improving the accuracy of the assessment results. By constructing an aligned data matrix, a causal relationship between physiological responses and external stimuli is established for each training event, making the assessment results interpretable. Finally, through error risk prediction and dynamic rating adjustment mechanisms, closed-loop management and pre-emptive warning of training effectiveness are achieved, effectively compensating for the blind spot of high-scoring but low-performance assessments.

[0029] As a preferred embodiment, the present invention further implements the above embodiments by limiting the specific dimensions of the multimodal physiological data and the acquisition methods of each indicator.

[0030] The multimodal physiological data includes data from multiple dimensions, including cardiovascular activity data, skin conductance activity data, and eye activity data.

[0031] In this embodiment of the invention, the ability of trainees is evaluated by collecting physiological data from multiple dimensions such as eye, cardiovascular, and skin conductance, which improves the comprehensiveness and effectiveness of data collection and lays a data foundation for subsequent calculation ability scoring.

[0032] Preferably, the cardiovascular activity data includes heart rate variability (HRV), the skin conductance activity data includes skin conductance increment, and the eye activity data includes pupillary oscillation frequency.

[0033] Heart rate variability (HRV) refers to the time variation between successive heartbeat cycles. It is a core indicator for assessing the balance of the autonomic nervous system. A decrease in HRV value indicates that the sympathetic nervous system is highly excited and the trainee is in a state of high-pressure stress.

[0034] Electrodermal activity (EDA) reflects changes in skin resistance caused by sweat gland secretion and is highly correlated with emotional arousal. By monitoring transient fluctuations in skin conductivity (GSR), EDA can be used to accurately capture moments of startle or emotional arousal in trainees.

[0035] Pupil oscillation frequency is of great significance in assessing cognitive or emotional changes in trainees. When the pupils are abnormally dilated and accompanied by an increase in the scanning frequency, the trainee is judged to be in a state of cognitive overload or fear.

[0036] The embodiments of the present invention integrate physiological data from multiple dimensions, such as heart rate variability, skin conductance increment, and pupillary oscillation frequency, which is beneficial for accurately assessing the psychological state of trainees.

[0037] Preferably, the heart rate variability index is calculated by monitoring the trainee's photoplethysmography (PPG) or electrocardiogram (ECG) data. The skin conductance increment index was calculated by monitoring the skin conductance activity of the trainee; The pupil oscillation frequency index is calculated by monitoring the pupil diameter data of the trainee.

[0038] In this embodiment of the invention, the physiological sensing device worn by the trainee includes a wristband or chest strap sensor that can be used to collect the trainee's PPG or ECG data and EDA data. It also includes a VR headset that integrates a near-infrared (NIR) eye-tracking camera for collecting the trainee's pupil diameter data.

[0039] For example, during cardiovascular activity monitoring, the trainee wears a chest strap ECG sensor or a wrist PPG sensor. The sensor continuously acquires ECG signals or photoplethysmography (PPG) signals at a sampling rate of 250 Hz or higher. The system performs a 0.5-50 Hz bandpass filter on the acquired raw signals to remove baseline drift and high-frequency noise. Hz is used to filter out baseline drift caused by breathing or exercise. Hz is used to filter out power frequency interference and high-frequency electromyographic noise. Then, an adaptive threshold peak detection algorithm is used to identify the R peak of the ECG signal or the peak of the PPG signal, calculate the time interval between adjacent peaks (RR interval), and further calculate the time-domain and frequency-domain characteristics of the RR interval. For example, the time-domain characteristic is RMSSD, which is the root mean square of the difference between adjacent RR intervals, and the frequency-domain characteristic is the LF / HF ratio, thus obtaining the heart rate variability (HRV) index. This index reflects the balance state of the trainee's autonomic nervous system; a decrease in HRV indicates sympathetic nerve excitation and that the trainee is in a state of stress.

[0040] During the monitoring of skin conductance activity, the EDA sensor built into the trainee's wristband uses a micro Siemens (Micro Siemens) sensor. The system continuously acquires skin conductance signals (EDA) in units of [unit missing], reflecting the trainee's emotional arousal and startle response. The system uses convex optimization or low-pass filtering algorithms to separate the raw EDA signal into slowly changing skin conductance level (SCL) and rapidly changing skin conductance response (SCR). SCL reflects overall arousal, while SCR reflects instantaneous stimulation. The peak amplitude and cumulative increment of the SCR waveform are extracted as the skin conductance increment index. Skin conductance increment sequence (unit missing) The micro-Siemens increment (i.e., the micro-Siemens increment) records the increase in skin conductance signal at each sampling point within the window relative to the start time of the window, reflecting the instantaneous net change in skin conductance caused by the event stimulus.

[0041] During eye movement monitoring, the near-infrared eye-tracking camera built into the VR headset continuously captures images of the trainee's eyes at a frame rate of over 60Hz. Image processing algorithms are used to extract the pupil contour and calculate the pupil diameter. After filtering the continuous pupil diameter time series to remove abrupt blinking changes, the system performs a Fast Fourier Transform (FFT) on the continuous pupil diameter time series, extracting the energy proportion in the 0.1-1Hz frequency band, and calculating the number of rapid pupil diameter expansion and contraction cycles per unit time as an indicator of pupil oscillation frequency. This indicator reflects the dynamic changes in the trainee's cognitive load and fear level.

[0042] The data from the above three dimensions are collected synchronously on the timeline, together forming multimodal physiological data.

[0043] By employing the technical means of this invention, the multidimensional composition and calculation method of physiological data are specifically defined. HRV, EDA, and pupillary oscillation frequency capture the psychological state of trainees from three dimensions: autonomic nervous system balance, emotional arousal, and cognitive load, respectively, forming a multidimensional quantitative representation of psychological stress and improving the comprehensiveness and robustness of the assessment. Furthermore, all indicators are based on mature physiological sensing technology, ensuring reliable data acquisition, reproducible calculations, and good engineering feasibility.

[0044] As a preferred embodiment, the present invention further implements the above embodiments based on any of the above embodiments, and limits the specific data structure and content of the event-physiology aligned data matrix.

[0045] The event-physiology aligned data matrix includes each event node, the timestamp of the event node, and a real-time value sequence of multimodal physiological data within the physiological data window corresponding to the event node; the physiological data window corresponding to the event node is a time window of a preset length truncated with the timestamp of the event node as the center.

[0046] In this embodiment of the invention, the system extracts physiological data windows for 3 seconds before and after each time point based on the timestamp corresponding to that time point, and binds and stores the real-time value sequences of HRV, skin conductance increment, and pupil oscillation frequency within the window with the event to form an aligned data matrix.

[0047] For example, taking the aforementioned military and police shooting simulation training scenario as an example, the preset event nodes and their timestamps in the VR scenario are shown in Table 1: Table 1 For each event node, the system extracts a 3-second window of physiological data before and after the event's timestamp (a total of 6 seconds). Taking event E001 as an example, the system extracts the real-time value sequence of all HRV, skin conductance increment, and pupillary oscillation frequency within the physiological data window formed by the 3 seconds before and after the timestamp from the physiological data stream. This data sequence, along with the event ID "E001", event type "encountering enemy attack", and timestamp "10:05:01.200", is stored as a single row in a matrix.

[0048] For example, the specific structure of the generated event-physiology aligned data matrix is ​​shown in Table 2 below: Table 2 It should be noted that the physiological data window length for different event nodes can be the same, for example, 3 seconds before and after. Alternatively, different window lengths can be preset according to the event type. For example, 5 seconds before and after the event can be used for high-intensity stimulation events such as explosions, and 2 seconds before and after the event can be used for low-intensity stimulation events such as voice commands. All of these fall within the protection scope of this invention.

[0049] By employing the technical means of this invention, the data structure of the aligned data matrix is ​​specifically defined. This matrix structure effectively organizes discrete event nodes and continuous physiological data streams in a structured manner, enabling the physiological response data of each event node to be independently indexed and retrieved, providing a clear data foundation for subsequent event-by-event feature value calculation. The windowing method centered on the event timestamp ensures the causal correlation of physiological responses. The captured physiological data window precisely covers the entire process before (baseline state) and after (stress response) the event stimulus, guaranteeing the objectivity and validity of the data. Furthermore, the matrix-based data organization facilitates subsequent parallel computing and batch processing, improving the system's data processing efficiency.

[0050] In a preferred embodiment, calculating the multimodal physiological feature values ​​of the trainee at each event node based on the event-physiological alignment data matrix includes: Based on the event-physiological alignment data matrix, feature extraction is performed on the real-time value sequence of multimodal physiological data within each physiological data window to obtain the multimodal physiological feature values ​​of the trainee at each event node.

[0051] In this embodiment of the invention, based on the type of physiological data, features such as the center value, maximum value, minimum value, and average value of the corresponding real-time value sequence are extracted as multimodal physiological feature values.

[0052] For example, the minimum value of the HRV sequence within the window is taken as the post-event stress HRV, the peak value of the EDA sequence is taken as the event-triggered skin conductance response value, and the maximum value of the pupil oscillation frequency sequence is taken as the event-triggered pupil dilation value, thereby obtaining the multimodal physiological characteristic values ​​under the event node.

[0053] For example, taking the event-physiology aligned data matrix in Table 2 above as an example, the system extracts features from the real-time value sequence within the physiological data window corresponding to each event node. Specifically: For the E001 event (encountering enemy attack), its physiological data window includes the HRV sequence [68,65,60,55,52,50,48,51], the skin conductance increment sequence [0.0,0.8,1.5,2.8,3.2,2.5], and the pupillary oscillation frequency sequence [0.2,0.3,0.5,0.8,1.2,1.5,1.3,0.9].

[0054] HRV Feature Extraction: The minimum value of 48 is extracted from the HRV sequence within the window, representing the instantaneous point of strongest post-event stress. The mean value of 51.2 of the sequence [55, 52, 50, 48, 51] in the second half of the window (after the event) is calculated as the stress HRV feature value of this event node. The lower the value, the stronger the stress.

[0055] Skin electrical activity increment feature extraction: The peak value of 3.2 is extracted from the skin electrical activity increment sequence within this window to obtain the skin point increment feature value. The higher the value, the stronger the degree of fright.

[0056] Pupil oscillation frequency feature extraction: The peak value of 1.5 is extracted from the pupil oscillation frequency sequence within this window to obtain the pupil oscillation frequency feature value. The higher the value, the heavier the cognitive load.

[0057] The same feature extraction operation was performed on events E002, E003, and E004 to obtain a set of multimodal physiological feature values ​​for each event node. After feature extraction, the system backfilled the extracted feature values ​​into the aligned data matrix as the data basis for subsequent calculation of psychological stress scores.

[0058] The technical means employed in this invention extracts features based on sequence data within a physiological data window, rather than just a single instantaneous value, thus ensuring statistical stability and noise resistance of the extracted feature values. The extracted feature values ​​(minimum, mean, peak, cumulative increment, etc. within the window) are all classic statistical quantities in the field of physiological signal analysis, with clear physical meanings, facilitating subsequent interpretation and modeling. The event-by-event, dimension-by-dimensional feature extraction method ensures that the evaluation indicators of each event node are independent of each other, providing accurate data support for the subsequent calculation of psychological stress scores.

[0059] As a preferred embodiment, the present invention further implements any of the above embodiments, and limits the specific implementation methods for calculating behavioral performance scores and psychological stress scores.

[0060] The step of calculating the trainee's behavioral performance score and psychological stress score at each event node based on the baseline value of the multimodal physiological data, the multimodal physiological feature value, and the behavioral response data includes: Based on the behavioral response data and preset standard behavioral rules, the trainee's behavioral performance score is calculated; wherein, the behavioral performance score is used to characterize the trainee's level of behavioral standardization. The trainee's psychological stress score is calculated based on the degree of deviation between the multimodal physiological characteristic values ​​and the baseline values ​​of the multimodal physiological data; wherein the psychological stress score is used to characterize the trainee's current psychological load and stress level.

[0061] Specifically, the behavioral performance score is calculated by the system based on the trainee's objective behavioral data before and after each event node, combined with preset standard behavioral rules.

[0062] Behavioral performance score The calculation formula is as follows: Taking the E001 incident (encountering enemy attack) in shooting simulation training as an example, the system records the trainee's behavioral data within 3 seconds after the incident, including shooting accuracy and reaction time. The behavioral performance is then categorized as follows: For example, a total of 5 shots were fired in the event window, with 4 hitting the target, resulting in a shooting accuracy of 80%; the time from the event trigger (enemy appearance) to the trainee completing the first shot, i.e., the reaction time, is 0.35 seconds; the preset weighting coefficients are: w1 = 0.6, w2 = 0.4.

[0063] The behavioral manifestations of the E001 event can be categorized as follows: Similarly, the system calculates behavioral performance scores for events E002, E003, and E004 respectively. The higher the behavioral performance score, the more standard the trainee's behavior and the better their operation.

[0064] The psychological stress score is obtained by comparing the physiological characteristic values ​​of each event node with the collected resting baseline values, calculating the degree of deviation, and then weighting and summing them.

[0065] Psychological stress The calculation formula is as follows: in, These are the weight coefficients of the AI ​​model, trained based on a large amount of historical training data, with HRV typically having the highest proportion. , These are the baseline and characteristic values ​​of HPV, respectively. , These are the baseline and characteristic values ​​of the skin electrical flux increment, respectively. , These are the baseline and characteristic values ​​of the pupil oscillation frequency, respectively.

[0066] For example, suppose the trainee's resting baseline value is: , , .

[0067] For event E001, the extracted physiological feature values ​​are: , , The weight coefficients for AI model pre-training are: , , .

[0068] The psychological stress of the E001 incident can be categorized as follows: =0.5×(65-48) / 65+0.3×(3.2 / 1.0)+0.2×(1.5-0.2) / 0.2=0.5×0.262+0.3×3.2+0.2×6.5= 0.131+ 0.96+1.3=2.391; Similarly, the system calculates psychological stress scores for events E002, E003, and E004 respectively. The higher the psychological stress score, the stronger the trainee's psychological load and stress level at the corresponding event node.

[0069] It is understood that the above scenarios and scenario data are merely examples, and in actual applications they are set and recorded according to the actual situation, and do not constitute a limitation of the present invention.

[0070] The technical means employed in this invention define the calculation methods for behavioral performance scores and psychological stress scores. The behavioral performance score is calculated based on the quantitative calculation of objective behavioral data, eliminating individual bias in subjective evaluations and ensuring consistency and reproducibility of the assessment results. The psychological stress score is calculated based on the deviation between multidimensional physiological characteristic values ​​and individualized benchmark values, fully considering individual differences among trainees and achieving personalized assessment for different individuals. The parallel calculation of the two scores provides a quantitative data foundation for subsequent cross-judgment of behavior and stress.

[0071] As a preferred embodiment, the present invention further implements any of the above embodiments, wherein the step of predicting error risk and rating proficiency based on the behavioral performance score and the psychological stress score includes: Based on the psychological stress sequence at each event node during the current simulation training task and the trainee's fatigue index, predict the trainee's error probability in subsequent simulation training tasks. When the probability of error exceeds a preset warning threshold, a preset alarm message is pushed. When the behavioral performance score is greater than a preset first scoring threshold and the psychological stress score is greater than a preset second scoring threshold, the step size for adjustment is calculated based on the psychological stress score and the error probability. The trainee's current proficiency rating is downgraded based on the downgrade step size to obtain the final proficiency rating after downgrading.

[0072] In this embodiment of the invention, the system provides a data basis for subsequent error probability prediction and rating downgrade based on real-time continuous behavioral performance scores and psychological stress scores.

[0073] The system monitors the trainee's blood oxygen saturation (SpO2) trend to characterize the trainee's fatigue level and obtain the complexity coefficient of the next simulated training task to be performed. ), combined with the current stage The time series data, and the probability of prediction errors.

[0074] A risk prediction model is built using exponential smoothing or LSTM networks. If a trainee's psychological recovery period (the time it takes for their stress score to return to baseline) is significantly longer than average after experiencing a high-stress event, and is accompanied by fatigue due to decreased blood oxygen levels (…), then… The system is substituted into the error probability formula: in, The Sigmoid activation function maps the results to 0-100%.

[0075] Output the percentage probability of failure for subsequent tasks ( Set an early warning threshold, for example, 75%. The system triggers a UI alarm, pushes prevention suggestions to the instructor's terminal, or dynamically adjusts the generated content in the MR / VR scene.

[0076] Furthermore, obtain the trainees' initial proficiency rating. For example, levels 1-5 are assigned by the system based on the student's historical average score, combined with the comprehensive behavioral performance score and comprehensive psychological stress score calculated (e.g., by averaging) based on the behavioral performance score sequence and psychological stress score sequence from this simulation training. , as well as .

[0077] A first scoring threshold S1 and a second scoring threshold S2 are set. The first scoring threshold is used to represent the standard score, and the second scoring threshold is used to represent the extremely high stress score. The following triggering condition is met, i.e., the local bureau... ,but Furthermore, when the highest pressure point aligns with a critical operational node, which is a typical case of high score but low ability or passing by luck, the proficiency rating will be downgraded.

[0078] Calculate the downsizing step size ( ): Where k is the adjustment coefficient, which is usually limited to a maximum single downgrade of no more than 1 level.

[0079] Calculate the final rating: (The result is rounded down).

[0080] Generate and output the final proficiency rating after the downgrade, and output a detailed training evaluation report including the reasons for the deduction (specific points where psychological qualities were not up to par).

[0081] Using the technical means of this invention, error risk prediction integrates three dimensions: the cumulative effect of psychological stress, the difficulty of the task itself, and physiological fatigue indicators. The prediction results are comprehensive and highly accurate, and can proactively intervene before errors occur, possessing early warning value. The rating downgrade mechanism accurately targets specific situations where behavior is good but stress is high, solving the problem of not being able to identify high scores but low abilities in traditional assessments. This ensures that proficiency ratings truly reflect the trainee's real ability under pressure, rather than just the operational results. Furthermore, the formula for calculating the downgrade step size considers both the degree of stress deviation and the probability of error, with the downgrade magnitude positively correlated with the risk level. The rating adjustment has a quantitative basis and interpretability, avoiding inconsistencies caused by subjective discretion.

[0082] See Figure 2This is a schematic diagram of the structure of a simulation training evaluation device provided in an embodiment of the present invention. The embodiment of the present invention provides a simulation training evaluation device 10, comprising: The data acquisition module 11 is used to collect baseline values ​​of the trainee's multimodal physiological data, as well as real-time values ​​and behavioral response data of the multimodal physiological data during the execution of a simulated training task containing preset event nodes; Data alignment module 12 is used to align the real-time values ​​of the trainee's multimodal physiological data with the event node in the time domain, based on the event node as the timestamp, and establish an event-physiological alignment data matrix; Feature extraction module 13 is used to calculate the multimodal physiological feature values ​​of the trainee at each event node based on the event-physiological alignment data matrix. The comprehensive scoring module 14 is used to calculate the trainee's behavioral performance score and psychological stress score at each event node based on the baseline value of the multimodal physiological data, the multimodal physiological feature value, and the behavioral response data. The risk prediction module 15 is used to predict error risk and skill level based on the behavioral performance score and the psychological stress score.

[0083] It should be noted that the simulation training and evaluation device provided in this embodiment of the invention is used to execute all the process steps of the simulation training and evaluation method in the above embodiment. The working principle and beneficial effect of the two are one-to-one, so they will not be described again.

[0084] See Figure 3 This is a schematic diagram of the structure of a simulation training and evaluation device provided in an embodiment of the present invention. The present invention also provides a simulation training and evaluation device 20, including a processor 21, a memory 22, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the simulation training and evaluation method as described in any of the above embodiments.

[0085] This invention also provides a computer-readable storage medium comprising a stored computer program, wherein the computer program, when running, controls the device containing the computer-readable storage medium to execute the simulation training and evaluation method as described in any of the above embodiments.

[0086] This invention also provides a computer program product, which includes a computer program or computer instructions. When the computer program or computer instructions are executed by a processor, they implement the simulation training and evaluation method as described in any of the above embodiments.

[0087] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0088] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A simulation training evaluation method, characterized in that, include: The baseline values ​​of the multimodal physiological data of the trainees are collected, as well as the real-time values ​​and behavioral response data of the multimodal physiological data during the execution of a simulated training task containing preset event nodes; Using the event node as a timestamp as a reference, the real-time values ​​of the trainee's multimodal physiological data are time-domain aligned with the event node to establish an event-physiological alignment data matrix; Based on the event-physiology alignment data matrix, calculate the multimodal physiological feature values ​​of the trainee at each event node; Based on the baseline values ​​of the multimodal physiological data, the multimodal physiological characteristic values, and the behavioral response data, the trainee's behavioral performance score and psychological stress score at each event node are calculated. Error risk prediction and proficiency rating are performed based on the behavioral performance score and the psychological stress score.

2. The simulation training evaluation method as described in claim 1, characterized in that, The multimodal physiological data includes data from multiple dimensions, including cardiovascular activity data, skin conductance activity data, and eye activity data. The cardiovascular activity data includes heart rate variability index, which is calculated by monitoring the photoplethysmography (PPI) or electrocardiogram (ECG) data of the trainees. The skin conductance activity data includes skin conductance increment index, which is calculated by monitoring the skin conductance activity of the trainee; The eye activity data includes a pupillary oscillation frequency index, which is calculated by monitoring the pupil diameter data of the trainee.

3. The simulation training evaluation method as described in claim 1 or 2, characterized in that, The event-physiology aligned data matrix includes each event node, the timestamp of the event node, and a real-time value sequence of multimodal physiological data within the physiological data window corresponding to the event node; the physiological data window corresponding to the event node is a time window of a preset length truncated with the timestamp of the event node as the center.

4. The simulation training evaluation method as described in claim 3, characterized in that, The step of calculating the multimodal physiological feature values ​​of the trainee at each event node based on the event-physiological alignment data matrix includes: Based on the event-physiological alignment data matrix, feature extraction is performed on the real-time value sequence of multimodal physiological data within each physiological data window to obtain the multimodal physiological feature values ​​of the trainee at each event node.

5. The simulation training evaluation method as described in claim 1, characterized in that, The step of calculating the trainee's behavioral performance score and psychological stress score at each event node based on the baseline value of the multimodal physiological data, the multimodal physiological feature value, and the behavioral response data includes: Based on the behavioral response data and preset standard behavioral rules, the trainee's behavioral performance score is calculated; wherein, the behavioral performance score is used to characterize the trainee's level of behavioral standardization. The trainee's psychological stress score is calculated based on the degree of deviation between the multimodal physiological characteristic values ​​and the baseline values ​​of the multimodal physiological data; wherein the psychological stress score is used to characterize the trainee's current psychological load and stress level.

6. The simulation training evaluation method as described in claim 1, characterized in that, The method of predicting error risk and rating proficiency based on the behavioral performance score and the psychological stress score includes: Based on the psychological stress sequence at each event node during the current simulation training task and the trainee's fatigue index, predict the trainee's error probability in subsequent simulation training tasks. When the probability of error exceeds a preset warning threshold, a preset alarm message is pushed. When the behavioral performance score is greater than a preset first scoring threshold and the psychological stress score is greater than a preset second scoring threshold, the step size for adjustment is calculated based on the psychological stress score and the error probability. The trainee's current proficiency rating is downgraded based on the downgrade step size to obtain the final proficiency rating after downgrading.

7. A simulation training and evaluation device, characterized in that, include: The data acquisition module is used to collect baseline values ​​of the trainee's multimodal physiological data, as well as real-time values ​​and behavioral response data of the multimodal physiological data during the execution of a simulated training task containing preset event nodes; The data alignment module is used to align the real-time values ​​of the trainee's multimodal physiological data with the event node in the time domain, based on the event node as the timestamp, and establish an event-physiological alignment data matrix. The feature extraction module is used to calculate the multimodal physiological feature values ​​of the trainee at each event node based on the event-physiological alignment data matrix. The comprehensive scoring module is used to calculate the trainee's behavioral performance score and psychological stress score at each event node based on the baseline value of the multimodal physiological data, the multimodal physiological feature value, and the behavioral response data. The risk prediction module is used to predict error risk and assess proficiency based on the behavioral performance score and the psychological stress score.

8. A simulation training and evaluation device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the simulation training evaluation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the simulation training and evaluation method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program or computer instructions, which, when executed by a processor, implement the simulation training and evaluation method as described in any one of claims 1 to 6.

Images