Virtual simulation practical training platform based on artificial intelligence adaptive learning path

Abstract

The invention relates to the technical field of intelligent teaching, in particular to a virtual simulation practical training platform based on an artificial intelligence self-adaptive learning path. According to the method, the fatigue state information is obtained by collecting the physiological behavior data and carrying out fatigue evaluation analysis on the physiological data, then the dynamic interface adjustment parameters are obtained by carrying out practical training interface display optimization according to the fatigue state information, then interface adjustment execution is carried out, the practical training interface is dynamically optimized, the fatigue occurrence rate is reduced, and the practical training efficiency is improved. The practical training efficiency and comfort are improved; a cooperative control module carries out cooperative control analysis on the fatigue state information to output an environment risk threshold parameter, carries out environment risk assessment analysis on the environment multimode data and the environment risk threshold parameter to obtain an environment risk state, then generates an adaptive reminding instruction based on the environment risk state, and executes the adaptive reminding instruction; risk-free, low-risk and high-risk graded reminding is achieved, the problem that closed practical training environment perception is lacked is avoided, and practical training safety is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of intelligent teaching technology, specifically to a virtual simulation training platform based on an artificial intelligence adaptive learning path. Background Technology

[0002] Virtual simulation training platforms, as core equipment in the field of intelligent teaching, are significant in breaking the limitations of time and space, cost constraints, and safety risks inherent in traditional training. They can simulate complex or high-risk scenarios such as mechanical maintenance and working at heights, avoiding safety issues caused by equipment wear and tear, limited space, and operational errors in physical training. They support repeated training and real-time feedback on operational effects, helping users quickly master practical skills, and are particularly suitable for skill-intensive fields with high practical requirements. Simultaneously, they reduce the cost of training consumables and equipment, achieving a balance between large-scale teaching and personalized training, becoming an indispensable key tool for vocational skills development. Currently, traditional virtual simulation training platforms, when using VR headsets for training, are prone to causing eye strain and dizziness, affecting the progress of training. Furthermore, during training, the use of closed-back headphones for an immersive experience often prevents users from perceiving their external environment, posing safety hazards. Summary of the Invention

[0003] This invention provides a virtual simulation training platform based on an artificial intelligence adaptive learning path to solve the above-mentioned technical problems.

[0004] The first aspect of this invention provides a virtual simulation training platform based on an artificial intelligence adaptive learning path, including a multimodal data acquisition module, a fatigue assessment module, a dynamic interface optimization module, a collaborative control module, and an environmental state perception module.

[0005] The multimodal data acquisition module collects physiological and behavioral data from the user and environmental multimodal data from the external environment. Specifically: Physiological signal data is collected from the user using a pre-set physiological signal acquisition unit. Training operation behavior data is collected from the operation logs of the operation unit. Scene visual data is collected from the visual images of the training. Physiological signal data, training operation behavior data, and scene visual data are then aggregated to obtain physiological behavior data. External sound signals are collected using a pre-set microphone array to obtain sound modal data. Video is collected using a pre-set camera module to obtain video modal data. Sound modal data and video modal data are then aggregated to obtain environmental multimodal data.

[0006] The fatigue assessment module is used to acquire physiological behavioral data and perform fatigue assessment analysis on the physiological data to obtain fatigue status information.

[0007] As a further improvement of the present invention, the specific evaluation process for fatigue assessment analysis of physiological data is as follows: The training period is divided based on a preset time interval. Physiological data of the user corresponding to the current time period is obtained. Physiological signal data, training operation behavior data and scene visual data are obtained based on the physiological data. Physiological fatigue values ​​are output based on the analysis of physiological signal data; operational fatigue values ​​are output based on the analysis of training operation behavior data; scene visual fatigue values ​​are output based on the analysis of scene visual data; and fatigue status information is output by comprehensively calculating and analyzing physiological fatigue values, operational fatigue values, and scene visual fatigue values.

[0008] Furthermore, physiological fatigue values ​​are output based on the analysis of physiological signal data, specifically as follows: Pupil parameters, heart rate signals, and skin electrical signals are obtained based on physiological signal data; the pupil diameter of a preset number of samples is obtained based on the pupil parameters; the average pupil diameter is obtained by summing the pupil diameters and calculating the average value; and the standard deviation of the pupil diameter is obtained by calculating the standard deviation of the pupil diameter by comparing the standard deviation of each pupil diameter with the average pupil diameter.

[0009] Obtain the current user's current training duration and the preset maximum training duration, and calculate the dynamic decay of the training duration using the preset formula. Calculate and output the dynamic attenuation coefficient for practical training. ;in These represent the current usage time and the maximum training time, respectively; sj is the preset attenuation coefficient with a value of 0.75, used to simulate the physiological law that for every 15 minutes increase in training time, the fatigue level corresponding to the same physiological parameters increases by 20%-25%.

[0010] Obtain the average pupil diameter of the user for the current time period and the pre-stored baseline pupil diameter of each user in a fatigue-free state; calculate and output the individual baseline correction value using a preset individual baseline correction calculation formula.

[0011] Through a preset multi-parameter nonlinear mapping formula Calculate and output the physiological fatigue value SL; where, All are preset weighting coefficients, with values ​​ranging from 1 to 10. ; It is the hyperbolic tangent function.

[0012] Furthermore, operational fatigue values ​​are output based on the analysis of practical training operation data, specifically as follows: Based on the training operation behavior data, operation deviation data, operation response data, and operation handle data are obtained; based on the operation deviation data, the distance difference between the actual operation position and the standard position is extracted, the distance difference corresponding to the preset number of operation times is obtained, and the average value is calculated and then marked as the operation deviation mean.

[0013] The response duration corresponding to the preset number of operation impacts is obtained by extracting the operation response data, and the standard deviation and mean of the operation response corresponding to the response duration are obtained by calculating the standard deviation and mean of the operation response; the ratio obtained by calculating the ratio of the standard deviation of the operation response to the mean of the operation response is marked as the response variation coefficient.

[0014] The jitter acceleration of the gyroscope is acquired based on the data from the control handle, and a preset jitter threshold is obtained. The number of times the jitter acceleration exceeds the jitter threshold during the current training period is counted and recorded as the total number of high-frequency jitters. The mean of the operation deviation, the coefficient of variation of the response, and the total number of high-frequency jitters are substituted into the preset Markov state transition formula. Calculate and output the operational fatigue value CZ; where, These are the mean operating deviation, the coefficient of variation of the response, and the total number of high-frequency flutters, respectively. These are the preset maximum thresholds for each parameter; The preset state dependency weights, i.e. Its value is and The more severe the fatigue state, the higher the weight is assigned to the combination of operating parameters; Let v be the Markov state transition probability, and let v be the operating state, which is divided into normal (v=1), slightly deteriorated (v=2), and severely deteriorated (v=3). The output is a Markov state chain model trained with preset historical operation states. A historical fatigue value decay memory accumulation term is set to simulate the characteristic of operational fatigue having short-term memory, but decaying slowly with short rests. These are the preset memory strength coefficient and operation attenuation coefficient, with values ​​of 0.45 and 0.12, respectively. This represents the time interval between the previous fatigue value and the current moment. This represents the fatigue value from the previous operation.

[0015] Furthermore, the scene visual fatigue value is output based on the analysis of scene visual data. Specifically, the pixel motion amplitude and scene brightness value of the current frame are extracted from the scene visual data; the average value of the pixel motion amplitude of each pixel in the current frame is calculated and marked as the pixel average amplitude; the regional brightness value and the scene average brightness value of the HDR region are extracted based on the scene brightness value, and the difference between the regional brightness value and the scene average brightness value is marked as the regional brightness deviation value; the product of the pixel average amplitude value and the regional brightness deviation value is marked as the scene visual fatigue value.

[0016] Furthermore, the fatigue status information is output by comprehensively calculating and analyzing physiological fatigue value, operational fatigue value, and scene visual fatigue value, specifically as follows: Physiological fatigue values, operational fatigue values, and scene visual fatigue values ​​are normalized and their values ​​are taken. Then, they are all input into the preset Choquet fuzzy integral formula. Calculate and output the comprehensive fatigue value Z; where, , Let represent the normalized values ​​of physiological fatigue, operational fatigue, and scene visual fatigue, respectively; h(x) is a monotonic function of the fuzzy measure, and the fatigue values ​​of each dimension are arranged in descending order, i.e. ; The fuzzy density function is obtained through optimization using a genetic algorithm.

[0017] When the overall fatigue value exceeds the preset overall fatigue threshold, the corresponding user is marked as a fatigued user; otherwise, fatigue status information is generated as no fatigue. The portion of the overall fatigue value exceeding the overall fatigue threshold is marked as a fatigue level value. The fatigue level value is divided into three fatigue over-limit intervals according to preset numerical intervals. The three fatigue over-limit intervals are arranged in ascending order and marked as mild fatigue interval, moderate fatigue interval, and severe fatigue interval, respectively. The fatigue level value of the fatigued user is matched with each fatigue over-limit interval. When the user is in the mild fatigue interval, fatigue status information is generated as mild fatigue; when the user is in the moderate fatigue interval, moderate fatigue is generated; and when the user is in the severe fatigue interval, severe fatigue is generated.

[0018] The dynamic interface optimization module is used to detect fatigue state information and optimize the display of the training interface to obtain dynamic interface adjustment parameters, and then perform interface adjustment. Its specific workflow is as follows: extract mild fatigue, moderate fatigue and severe fatigue based on fatigue state information, and mark them as fatigue information to be adjusted; obtain the pre-set interface optimization adjustment library, that is, the preset dynamic interface adjustment parameters corresponding to each fatigue information to be adjusted; match each fatigue information to be adjusted with the interface optimization adjustment library and output the corresponding dynamic interface adjustment parameters.

[0019] The collaborative control module is used to acquire fatigue state information, perform collaborative control analysis, output environmental risk threshold parameters, and feed them back to the environmental state perception module. Its specific workflow is as follows: Fatigue state information is identified as mild, moderate, and severe fatigue. When the fatigue state is mild, preset environmental hazard trigger thresholds and environmental hazard upper limit thresholds are acquired. When the fatigue state is moderate, both the environmental hazard trigger threshold and environmental hazard upper limit threshold are reduced by a preset threshold adjustment amplitude. When the fatigue state is severe, both the environmental hazard trigger threshold and environmental hazard upper limit threshold are reduced by a factor of two. The environmental risk threshold parameters are then obtained by combining the environmental hazard trigger thresholds and environmental upper limit thresholds corresponding to each fatigue state.

[0020] The environmental status perception module is used to acquire multi-mode environmental data and environmental risk threshold parameters, perform environmental risk assessment and analysis on the multi-mode environmental data and environmental risk threshold parameters to obtain the environmental risk status, and then generate and execute the adaptive reminder instructions based on the environmental risk status.

[0021] As a further improvement of the present invention, environmental risk assessment and analysis are performed on environmental multi-modal data and environmental risk threshold parameters. The specific analysis process is as follows: Sound modal data and video modal data are obtained based on environmental multimodal data. Voiceprint signals are extracted from the sound modal data, and a pre-defined voiceprint signal library is obtained, which classifies voiceprints into dangerous voiceprint features and ordinary noise voiceprint features. The current voiceprint signal is matched with the pre-defined voiceprint signal library to obtain the voiceprint matching degree between each voiceprint signal and the dangerous voiceprint features. The duration of dangerous voiceprints is also calculated, and a pre-defined voiceprint danger output formula is used. The voiceprint hazard value was calculated. Its value is constrained to [0, 1]; where, These are voiceprint matching accuracy and voiceprint duration, respectively. This is a preset saturation duration, used to saturate the duration of the voiceprint. Exceed according to count.

[0022] Object recognition targets are obtained based on video modal data, and their movement trajectories are acquired. These trajectories are then matched against a pre-defined hazardous target database to identify hazardous targets. The shortest distance between each hazardous target and the user is calculated, and the total number of hazardous targets is determined. A saturation limit is set for the total number of hazardous targets; if the limit is exceeded, the calculation is based on the saturation limit. A pre-defined video hazard output formula is then used. Calculate and output video hazard values Its value is constrained to [0, 1]; where, These are the shortest distance to the target and the total number of dangerous targets, respectively. This is the distance attenuation coefficient.

[0023] The comprehensive environmental hazard value is obtained by weighted fusion of voiceprint hazard values ​​and video hazard values, i.e., by formula. Calculate and output the comprehensive environmental hazard value The environmental risk threshold parameters are matched and analyzed with the comprehensive environmental hazard value to output the environmental risk status.

[0024] Furthermore, the environmental risk threshold parameters are matched and analyzed with the comprehensive environmental hazard value to output the environmental risk status. The specific workflow is as follows: Based on the environmental risk threshold parameters, the environmental hazard trigger threshold and the environmental hazard upper limit threshold are obtained and denoted as follows: The overall environmental hazard value meets the requirements. At that time, the environmental risk status was no risk; the comprehensive environmental hazard value met the requirements. At that time, the environmental risk status was low; the comprehensive environmental hazard value met the requirements. At that time, the environmental risk status was high.

[0025] The beneficial effects of the technical solution provided by this invention compared with the prior art are as follows: 1. This invention collects physiological behavioral data from users, performs fatigue assessment and analysis on the physiological data to obtain fatigue state information, and then optimizes the training interface display based on the fatigue state information to obtain dynamic interface adjustment parameters. The interface adjustment is then executed to dynamically optimize the training interface, reduce the incidence of fatigue, and improve training efficiency and comfort.

[0026] 2. This invention uses a collaborative control module to analyze fatigue state information and output environmental risk threshold parameters. It then performs environmental risk assessment and analysis on multi-mode environmental data and environmental risk threshold parameters to obtain the environmental risk status. Based on the environmental risk status, it generates and executes appropriate reminder instructions, achieving graded reminders for no risk, low risk, and high risk. This avoids the problem of lack of awareness in closed training environments and ensures training safety. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The following drawings are not deliberately drawn to scale according to the actual size, but are intended to show the main idea of ​​this application.

[0028] Figure 1 This is a schematic diagram of the virtual simulation training platform based on artificial intelligence adaptive learning path of the present invention. Figure 2This is a schematic diagram illustrating the specific evaluation process for fatigue assessment analysis of physiological data according to the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1-2 In this embodiment of the invention, the virtual simulation training platform based on an artificial intelligence adaptive learning path includes: The multimodal data acquisition module collects physiological and behavioral data from the user and environmental data from the external environment to obtain environmental multimodal data, specifically: Physiological signal data is collected from the user based on a preset AI adaptive physiological signal acquisition unit (such as an eye tracker and wrist heart rate acquisition device built into the VR headset). Training operation behavior data is collected based on the operation log of the operation unit. Scene visual data is collected from the visual images of the training. Physiological signal data, training operation behavior data and scene visual data are combined to obtain physiological behavior data.

[0031] Sound modal data is obtained by collecting external sound signals through a preset microphone array, and video modal data is obtained by collecting video through a preset camera module. The sound modal data and video modal data are then combined to obtain environmental multimodal data.

[0032] The fatigue assessment module acquires physiological behavioral data and performs fatigue assessment analysis on the physiological data to obtain fatigue status information.

[0033] The specific assessment process for fatigue assessment analysis of physiological data is as follows: The training period is divided based on a preset time interval. Physiological data of the user corresponding to the current time period is obtained. Physiological signal data, training operation behavior data and scene visual data are obtained based on the physiological data. The physiological fatigue value is output based on the analysis of physiological signal data. Specifically, pupil parameters, heart rate signals, and skin conductance signals are obtained from the physiological signal data. A preset number of pupil diameters are obtained from the pupil parameters. The average pupil diameter is obtained by summing the values ​​of each pupil diameter and calculating the average. The standard deviation of the pupil diameter is then calculated by comparing the standard deviation of each pupil diameter with the average pupil diameter, i.e., by using the formula... Calculate and output the standard deviation of pupil diameter. ;in, Let be the pupil diameter at the i-th sampling point. To preset the number of samples, The average pupil diameter is used; based on the heart rate signal, the low-frequency component heart rate energy and high-frequency component heart rate energy within the preset acquisition time period are obtained, and the ratio of the low-frequency component heart rate energy to the high-frequency component heart rate energy is calculated to obtain the heart rate energy ratio; the LF / HF ratio does not reflect the activity of a single nerve branch, but directly quantifies the sympathetic-vagal balance state of the autonomic nervous system through the energy ratio relationship between LF and HF.

[0034] High-frequency energy (HF): The frequency range is usually defined as 0.15-0.4Hz. Its physiological source depends entirely on the vagus nerve (the inhibitory branch of the autonomic nervous system, responsible for slowing down the heart rate and reducing metabolism). The stronger the vagus nerve activity, the higher the HF energy. For example, when the human body is in a relaxed, awake, and resting state, the vagus nerve dominates heart rate regulation, the HF value increases significantly, and the heart rate fluctuations are more regular. Low-frequency energy (LF): The frequency range is usually defined as 0.04-0.15Hz. Its physiological source is the joint regulation of the sympathetic and vagus nerves (mainly the sympathetic nerve, which is the excitatory branch of the autonomic nervous system, responsible for accelerating the heart rate and increasing metabolism). An increase in LF energy usually means that the sympathetic nerve activity is enhanced. For example, when the human body is in a state of stress, exercise, or fatigue, the excitability of the sympathetic nerve increases, the LF value increases accordingly, and the heart rate fluctuations are more intense.

[0035] Obtain the current user's current training duration and the preset maximum training duration, and calculate the dynamic decay of the training duration using the preset formula. Calculate and output the dynamic attenuation coefficient for practical training. ;in These represent the current usage time and the maximum training time, respectively; sj is the preset attenuation coefficient, with a value of 0.75, used to simulate the physiological pattern that for every 15 minutes increase in training time, the fatigue level corresponding to the same physiological parameters increases by 20%-25%, such as... =60min, =90min, =exp(0.75×60 / 90)=1.65, compared to =65% higher at 15 minutes.

[0036] Obtain the average pupil diameter of the user for the current time period and the pre-stored baseline pupil diameter of each user in a fatigue-free state; then calculate the correction using a preset individual baseline formula. Calculate and output individual baseline correction values ;in, These represent the average pupil diameter over the time period and the baseline pupil diameter, respectively; gt is a preset correction coefficient with a value of 0.4, used to address the issue of individual fatigue tolerance decreasing with training duration. For example, if the average pupil diameter increases in the later stages of training, the individual baseline correction value further improves the final result, making it more consistent with the actual fatigue state.

[0037] Through a preset multi-parameter nonlinear mapping formula Calculate and output the physiological fatigue value SL; where, All are preset weighting coefficients, with values ​​ranging from 1 to 10. ; It is the hyperbolic tangent function.

[0038] The analysis and output of operational fatigue values ​​based on training operation behavior data are as follows: Operation deviation data, operation response data, and operation handle data are obtained from the training operation behavior data; the distance difference between the actual operation position and the standard position is extracted based on the operation deviation data, the distance difference corresponding to the preset number of operation times is obtained, and the average value is calculated and then marked as the average operation deviation value.

[0039] The response duration corresponding to the preset number of operation impacts is obtained by extracting the operation response data, and the standard deviation and mean of the operation response corresponding to the response duration are obtained by calculating the standard deviation and mean of the operation response; the ratio obtained by calculating the ratio of the standard deviation of the operation response to the mean of the operation response is marked as the response variation coefficient.

[0040] The jitter acceleration of the gyroscope is acquired based on the data from the control handle, and a preset jitter threshold is obtained. The number of times the jitter acceleration exceeds the jitter threshold during the current training period is counted and recorded as the total number of high-frequency jitters. The mean of the operation deviation, the coefficient of variation of the response, and the total number of high-frequency jitters are substituted into the preset Markov state transition formula. Calculate and output the operational fatigue value CZ; where, These are the mean operating deviation, the coefficient of variation of the response, and the total number of high-frequency flutters, respectively. These are the preset maximum thresholds for each parameter; The preset state dependency weights, i.e. Its value is and The more severe the fatigue state, the higher the weight is assigned to the combination of operating parameters; Let v be the Markov state transition probability, and let v be the operating state, which is divided into normal (v=1), slightly deteriorated (v=2), and severely deteriorated (v=3). The Markov state chain model, trained by pre-set historical operating states, outputs the probability of transitioning from normal to slightly deteriorated state, P(2∣1). The state transition matrix is ​​calculated based on historical operating data, integrating fatigue calculation into the logic of operating state evolution. This differs from the direct mapping of existing static parameters. For example, when the average operating deviation reaches 2mm, if the preceding state is normal, P(2∣1)=0.6; if the preceding state is slightly deteriorated, P(3∣2)=0.7, reflecting the continuity of the state. A historical fatigue value decay memory accumulation term is set to simulate the characteristic of operational fatigue having short-term memory, but decaying slowly with short rests. These are the preset memory strength coefficient and operation attenuation coefficient, with values ​​of 0.45 and 0.12, respectively. This represents the time interval between the previous fatigue value and the current moment. This is the fatigue value from the previous operation; for example... =0.6. After an interval of 10 seconds, the cumulative term of historical fatigue value decay memory is 0.45×0.6×exp(−0.12×10)=0.12, which avoids the unreasonableness of the fatigue value suddenly being cleared to zero in the existing model.

[0041] The scene visual fatigue value is output based on scene visual data analysis. Specifically, it is as follows: extract the pixel motion amplitude and scene brightness value of the current frame from the scene visual data; calculate the average value of the pixel motion amplitude of each pixel in the current frame and mark it as the pixel average amplitude; extract the regional brightness value and scene average brightness value of the HDR region (high-brightness region defined by AI visual analysis, i.e., brightness ≥1200 nit) based on the scene brightness value, mark the difference between the regional brightness value and the scene average brightness value as the regional brightness deviation value; and mark the product of the pixel average amplitude and the regional brightness deviation value as the scene visual fatigue value.

[0042] The fatigue status information is output by comprehensively calculating and analyzing physiological fatigue value, operational fatigue value, and scene visual fatigue value. Specifically: Physiological fatigue values, operational fatigue values, and scene visual fatigue values ​​are normalized and their values ​​are taken. Then, they are all input into the preset Choquet fuzzy integral formula. Calculate and output the comprehensive fatigue value Z; where, , Let represent the normalized values ​​of physiological fatigue, operational fatigue, and scene visual fatigue, respectively; h(x) is a monotonic function of the fuzzy measure, and the fatigue values ​​of each dimension are arranged in descending order, i.e. To ensure that the order of points is in line with the principle of prioritizing the integration of important dimensions; The density function is obtained through optimization using a genetic algorithm. For example, in the current mechanical maintenance training, =0.4, =0.35, =0.25; =0.65, =0.55, =0.5, =1; For subset The fuzzy measure value is used to reflect the overall importance of a subset.

[0043] When the overall fatigue value exceeds the preset overall fatigue threshold, the corresponding user is marked as a fatigued user; otherwise, fatigue status information is generated as no fatigue. The portion of the overall fatigue value exceeding the overall fatigue threshold is marked as a fatigue level value. The fatigue level value is divided into three fatigue over-limit intervals according to preset numerical intervals. The three fatigue over-limit intervals are arranged in ascending order and marked as mild fatigue interval, moderate fatigue interval, and severe fatigue interval, respectively. The fatigue level value of the fatigued user is matched with each fatigue over-limit interval. When the user is in the mild fatigue interval, fatigue status information is generated as mild fatigue; when the user is in the moderate fatigue interval, moderate fatigue is generated; and when the user is in the severe fatigue interval, severe fatigue is generated.

[0044] The dynamic interface optimization module detects fatigue state information and optimizes the training interface display to obtain dynamic interface adjustment parameters, which are then used to perform interface adjustments. Specifically, it extracts mild, moderate, and severe fatigue based on fatigue state information and marks them as fatigue information to be adjusted. It then obtains a pre-set interface optimization adjustment library, which contains preset dynamic interface adjustment parameters corresponding to each fatigue information to be adjusted. Finally, it matches each fatigue information to be adjusted with the interface optimization adjustment library and outputs the corresponding dynamic interface adjustment parameters. For example, for mild fatigue, only parameters related to visual load are optimized, such as reducing contrast in high-brightness areas and reducing... Non-core interface elements (such as hiding the history operation record panel); Moderate fatigue: Overlay display parameter optimization, such as reducing the field of view (FOV) (from 120° to 90°), improving the rendering accuracy of the gaze point area (degrading the edge area to reduce the computing load), and adjusting the virtual focal plane (simulating variable zoom through algorithms to alleviate VAC conflicts); Severe fatigue: Trigger progressive rest guidance, such as switching the training scene to a low-stimulation virtual rest space (blue background, low dynamic range), while pushing lightweight theoretical knowledge points related to the current training (such as part structure diagrams) to avoid completely interrupting the learning path.

[0045] The collaborative control module acquires fatigue state information, performs collaborative control analysis, outputs environmental risk threshold parameters, and feeds them back to the environmental state perception module. Specifically, it identifies fatigue state information as mild, moderate, and severe fatigue. When the fatigue state is mild, it acquires preset environmental hazard trigger thresholds and environmental hazard upper limit thresholds. When the fatigue state is moderate, it reduces both the environmental hazard trigger threshold and the environmental hazard upper limit threshold by a preset threshold adjustment range. When the fatigue state is severe, it reduces both the environmental hazard trigger threshold and the environmental hazard upper limit threshold by a factor of two. The environmental risk threshold parameters are then obtained by combining the environmental hazard trigger thresholds and environmental upper limit thresholds corresponding to each fatigue state.

[0046] The environmental status perception module acquires multi-modal environmental data and environmental risk threshold parameters, and performs environmental risk assessment and analysis on the multi-modal environmental data and environmental risk threshold parameters to obtain the environmental risk status. Then, based on the environmental risk status, it generates and executes the adaptive reminder instructions. That is, when the environmental risk status is no risk, no instruction is generated; when the environmental risk status is low risk, the adaptive reminder instruction generated is a corner pop-up reminder; when the environmental risk status is high risk, the adaptive reminder instruction generated is a pop-up, vibration and sound warning.

[0047] The specific analysis process for conducting environmental risk assessment and analysis on multi-modal environmental data and environmental risk threshold parameters is as follows: Sound modal data and video modal data are obtained based on environmental multimodal data. Voiceprint signals are extracted from the sound modal data, and a pre-defined voiceprint signal library is obtained, which categorizes voiceprints into dangerous voiceprint features (such as equipment collision sounds, sharp shouts, and fire alarm sounds) and ordinary noise voiceprint features (such as air conditioner sounds and footsteps). The current voiceprint signal is matched with the pre-defined voiceprint signal library to obtain the voiceprint matching degree between each voiceprint signal and the dangerous voiceprint features. The duration of dangerous voiceprints is also calculated, and a pre-defined voiceprint danger output formula is used. The voiceprint hazard value was calculated. Its value is constrained to [0, 1]; where, These are voiceprint matching accuracy and voiceprint duration, respectively. This is a preset saturation duration, used to saturate the duration of the voiceprint. Exceed according to The risk of sound is quickly quantified by linearly summing the dangerous voiceprint matching degree (weight 0.6) and normalized duration (weight 0.4, saturation at 5s) within the [0,1] interval.

[0048] Object recognition targets are obtained based on video modal data, and their movement trajectories are acquired. These trajectories are then matched against a pre-defined database of dangerous targets (the danger determination rules are based on targets that have entered a pre-defined warning range and whose movement trajectories tend towards the user's object recognition targets). Dangerous targets are identified by obtaining the shortest distance between each dangerous target and the user's target, and the total number of dangerous targets is calculated. A saturation limit is set for the total number of dangerous targets; targets exceeding this limit are counted as such. For example, if the saturation limit is 3, the maximum number of dangerous targets is 3; targets exceeding this limit are counted as 3. A pre-defined video hazard output formula is then used. Calculate and output video hazard values Its value is constrained to [0, 1]; where, These are the shortest distance to the target and the total number of dangerous targets, respectively. This is the distance attenuation coefficient. The distance attenuation coefficient (inversely proportional) is multiplied by the normalized number of targets (3 saturation), constrained in the [0,1] interval, to quickly quantify video risk (close range + multiple targets have higher risk).

[0049] The comprehensive environmental hazard value is obtained by weighted fusion of voiceprint hazard values ​​and video hazard values, i.e., by formula. Calculate and output the comprehensive environmental hazard value Based on the environmental risk threshold parameters, the environmental hazard trigger threshold and the environmental hazard upper limit threshold are obtained and denoted as follows: The overall environmental hazard value meets the requirements. At that time, the environmental risk status was no risk; the comprehensive environmental hazard value met the requirements. At that time, the environmental risk status was low; the comprehensive environmental hazard value met the requirements. At that time, the environmental risk status was high.

[0050] To facilitate calculations, all index data involved in the calculations in this embodiment of the invention have undergone data preprocessing, such as normalization or standardization, to eliminate the influence of dimensions. Specific methods for eliminating the influence of dimensions are well-known to those skilled in the art and are not limited here.

[0051] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A virtual simulation training platform based on an artificial intelligence adaptive learning path, characterized in that, include: The multimodal data acquisition module collects physiological and behavioral data from the user and environmental multimodal data from the external environment. The fatigue assessment module is used to acquire physiological and behavioral data and perform fatigue assessment analysis to obtain fatigue status information. The dynamic interface optimization module is used to detect fatigue state information and optimize the display of the training interface to obtain dynamic interface adjustment parameters, and then perform interface adjustment. The collaborative control module is used to acquire fatigue state information, perform collaborative control analysis, output environmental risk threshold parameters, and feed them back to the environmental state perception module. The environmental status perception module is used to acquire multi-modal environmental data and environmental risk threshold parameters, perform environmental risk assessment and analysis to obtain the environmental risk status, and then generate and execute adaptive reminder instructions based on the environmental risk status.

2. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 1, characterized in that, The specific assessment process for obtaining fatigue state information by acquiring physiological behavioral data and performing fatigue assessment analysis is as follows: The training period is divided based on a preset time interval. Physiological data of the user corresponding to the current time period is obtained. Physiological signal data, training operation behavior data and scene visual data are obtained based on the physiological data. Physiological fatigue values ​​are output based on the analysis of physiological signal data. Analyze and output operational fatigue values ​​based on training operation behavior data; Analyze and output scene visual fatigue values ​​based on scene visual data; The fatigue status information is output by comprehensively calculating and analyzing physiological fatigue value, operational fatigue value and scene visual fatigue value.

3. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 2, characterized in that, The specific steps for analyzing and outputting physiological fatigue values ​​based on physiological signal data are as follows: Pupil parameters, heart rate signals, and skin conductance signals are obtained based on physiological signal data; the pupil diameter of a preset number of samples is obtained based on the pupil parameters; the average pupil diameter is obtained by summing the pupil diameters and calculating the average value; and the standard deviation of the pupil diameter is obtained by calculating the standard deviation of the pupil diameter by comparing the standard deviation of each pupil diameter with the average pupil diameter. Based on the heart rate signal, the low-frequency component heart rate energy and high-frequency component heart rate energy within the preset acquisition time period are obtained, and the ratio of the low-frequency component heart rate energy to the high-frequency component energy is calculated to obtain the heart rate energy ratio. Obtain the current training duration and the preset maximum training duration of the current user, and calculate and output the dynamic training duration decay coefficient using the preset dynamic training duration decay calculation formula; Obtain the average pupil diameter of the user in the current time period and the pre-stored baseline pupil diameter of each user in a fatigue-free state; The individual baseline correction value is calculated and output using a preset individual baseline correction calculation formula; Physiological fatigue values ​​are calculated and output using a preset multi-parameter nonlinear mapping formula.

4. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 3, characterized in that, The specific steps for analyzing and outputting operational fatigue values ​​based on training operation behavior data are as follows: Based on the training operation behavior data, operation deviation data, operation response data, and operation handle data are obtained; based on the operation deviation data, the distance difference between the actual operation position and the standard position is extracted, the distance difference corresponding to the preset number of operation times is obtained, and the average value is calculated and then marked as the operation deviation mean. Based on the operation response data, the response duration corresponding to the preset number of operation impacts is obtained, and the standard deviation and mean of the operation response corresponding to the response duration are obtained by standard deviation calculation and mean calculation. The ratio obtained by calculating the ratio of the standard deviation of the operational response to the mean of the operational response is denoted as the coefficient of variation of the response. The jitter acceleration of the gyroscope is collected based on the data from the control handle, and a preset jitter threshold is obtained. The number of times the jitter acceleration exceeds the jitter threshold during the current training period is counted and recorded as the total number of high-frequency jitters. The mean of the operation deviation, the coefficient of variation of the response, and the total number of high-frequency jitters are substituted into the Markov state transition formula to output the operation fatigue value.

5. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 4, characterized in that, The process of analyzing and outputting scene visual fatigue value based on scene visual data specifically involves: extracting the pixel motion amplitude and scene brightness value of the current frame based on the scene visual data; calculating the average value of the pixel motion amplitude of each pixel in the current frame and marking it as the average pixel amplitude. Based on the scene brightness value, extract the regional brightness value and the scene average brightness value of the HDR region. The difference between the regional brightness value and the scene average brightness value is marked as the regional brightness deviation value. The product of the pixel average amplitude value and the regional brightness deviation value is marked as the scene visual fatigue value.

6. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 5, characterized in that, The process of comprehensively calculating and analyzing physiological fatigue values, operational fatigue values, and scene visual fatigue values ​​to output fatigue state information specifically involves: The physiological fatigue value, operational fatigue value, and scene visual fatigue value are normalized and their values ​​are taken. Then, they are all input into the preset Choquet fuzzy integral formula to calculate and output the comprehensive fatigue value. When the overall fatigue value exceeds the preset overall fatigue threshold, the corresponding user is marked as a fatigued user; otherwise, fatigue status information is generated as no fatigue. The portion of the comprehensive fatigue value that exceeds the comprehensive fatigue threshold is marked as the fatigue level value. The fatigue level value is divided into three fatigue over-limit intervals according to a preset numerical interval. The three fatigue over-limit intervals are arranged in ascending order and marked as mild fatigue interval, moderate fatigue interval, and severe fatigue interval, respectively. The fatigue level of the fatigued user is matched with each fatigue over-limit range. When the user is in the mild fatigue range, the fatigue status information is generated as mild fatigue; when the user is in the moderate fatigue range, the fatigue status information is generated as moderate fatigue. When the body is in the severe fatigue range, severe fatigue is generated.

7. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 6, characterized in that, The dynamic interface optimization module is used to detect fatigue state information and optimize the display of the training interface to obtain dynamic interface adjustment parameters, and then perform interface adjustment. Specifically, it does so as follows: Based on fatigue state information, mild fatigue, moderate fatigue, and severe fatigue are extracted and marked as fatigue information to be adjusted. A pre-set interface optimization adjustment library is obtained, which contains preset dynamic interface adjustment parameters corresponding to each fatigue information to be adjusted. Each fatigue information to be adjusted is matched with the interface optimization adjustment library to output the corresponding dynamic interface adjustment parameters.

8. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 7, characterized in that, The collaborative control module acquires fatigue state information, performs collaborative control analysis, outputs environmental risk threshold parameters, and feeds them back to the environmental state perception module. Specifically: The fatigue state information is identified as mild fatigue, moderate fatigue and severe fatigue. When the fatigue state information is mild fatigue, the preset environmental hazard trigger threshold and environmental hazard upper limit threshold are obtained. When the fatigue status information is moderate fatigue, both the environmental hazard trigger threshold and the environmental hazard upper limit threshold will be reduced by the preset threshold adjustment value; When the fatigue state information is severe fatigue, the threshold adjustment amplitude of both the environmental hazard trigger threshold and the environmental hazard upper limit threshold is reduced by a factor of two. The environmental risk threshold parameter is obtained by combining the environmental hazard trigger threshold and environmental upper limit threshold corresponding to each fatigue state information.

9. The virtual simulation training platform based on artificial intelligence adaptive learning path according to claim 8, characterized in that, The specific analysis process for obtaining the environmental risk status through environmental risk assessment analysis is as follows: Sound modal data and video modal data are obtained based on environmental multimodal data; voiceprint signals are extracted based on sound modal data, and a pre-set voiceprint signal library is obtained, which divides voiceprints into dangerous voiceprint features and ordinary noise voiceprint features; the current voiceprint signal is matched with the pre-set voiceprint signal library to obtain the voiceprint matching degree between each voiceprint signal and dangerous voiceprint features, and the voiceprint duration of dangerous voiceprints is counted. The voiceprint danger value is calculated by using a pre-set voiceprint danger output formula. Based on video modal data, object recognition targets are obtained, and their movement trajectories are acquired. These trajectories are then matched with a pre-defined hazardous target database to identify hazardous targets. The shortest distance between each hazardous target and the user's target is obtained, and the total number of hazardous targets is calculated. A saturation limit for the total number of hazardous targets is set, and if the limit is exceeded, the calculation is based on the saturation limit. The video hazard value is then calculated and output using a pre-defined video hazard output formula. The comprehensive environmental hazard value is obtained by weighted fusion of voiceprint hazard value and video hazard value; The environmental risk threshold parameter is matched with the comprehensive environmental hazard value to output the environmental risk status.

10. The virtual simulation training platform based on an artificial intelligence adaptive learning path according to claim 9, characterized in that, The specific workflow for matching and analyzing environmental risk threshold parameters with comprehensive environmental hazard values ​​to output environmental risk status is as follows: Based on the environmental risk threshold parameters, the environmental hazard trigger threshold and the environmental hazard upper limit threshold are obtained and denoted as follows: ; When the comprehensive environmental hazard value satisfy Under certain conditions, the environmental risk status is determined to be risk-free. When the comprehensive environmental hazard value satisfy Under certain conditions, the environmental risk status is determined to be low risk. When the comprehensive environmental hazard value satisfy Under certain conditions, the environmental risk status is determined to be high risk.

Images