Flight training regulation and control method and system

Abstract

The invention relates to the field of flight training data processing, in particular to a flight training regulation and control method and system, and the method comprises the steps: collecting a physiological signal, an electroencephalogram signal and a visual signal of a pilot, and generating comprehensive feature data through multi-modal feature fusion; time dependence analysis is carried out through a recurrent neural network, a comprehensive pressure trend is generated in combination with historical pressure data, the psychological pressure of the pilot is further evaluated through a deep learning model, and the training intensity is adjusted based on real-time feedback data; in the virtual training, the situation is dynamically adjusted based on reinforcement learning, and a personalized training scheme is generated in combination with personalized analysis, so that the training effect of the pilot is optimized; according to the invention, comprehensive monitoring and personalized training regulation and control of the physiological state and psychological stress of the pilot are realized, and the adaptability and safety of flight training are improved.

Description

Technical Field

[0001] The present invention relates to the field of flight training data processing, and in particular to a flight training control method and system. Background Art

[0002] Through effective flight training control, the emergency response ability and psychological quality of pilots can be improved, ensuring the safety and operation accuracy of pilots in complex flight missions. Existing flight training control systems usually adopt safety management methods based on visual technology, and conduct safety intervention by monitoring the pilot's physiological signals (such as heart rate). However, the existing technology (Chinese invention patent, publication number: CN118247494B, name: A flight training safety management system based on visual technology) mainly relies on taking pictures of the pilot's face to obtain the heart rate, and using rPPG technology (remote optical plethysmography technology) to analyze the pixel changes in the image to detect the heart rate. Traditional technologies use threshold segmentation and other methods to obtain the region of interest, but these technologies require calculations of all pixels in the color image, resulting in slow calculation speeds and difficulty in achieving fast and accurate heart rate detection, thereby affecting the timeliness of safety management of flight training. Summary of the invention

[0003] In view of the many problems existing in the above-mentioned prior art, the present invention provides a flight training control method and system, which realizes the all-round dynamic control of flight training through technical means such as multimodal feature fusion, time series analysis, psychological stress assessment and personalized training optimization. Through the dynamic interaction of multimodal data and the joint application of deep learning algorithms, real-time monitoring and adjustment of the pilot's physiological state and psychological stress are achieved, and personalized optimization is performed in a virtual situation to improve the safety and effect of training.

[0004] A flight training control method comprises the following steps: Collect the pilot's first physiological signal data, second EEG signal data and visual signal data, and perform preprocessing and feature extraction on the collected data to generate corresponding feature data; fuse all feature data to generate comprehensive feature data; Perform interactive calculation of multimodal features on the comprehensive feature data to generate interactive feature data; perform time-dependent analysis on the interactive feature data in combination with historical pressure change data to generate comprehensive pressure trend data; calculate the psychological stress index based on the interactive feature data and comprehensive pressure trend data, and generate real-time feedback parameter data and training intensity adjustment data; Based on the training intensity adjustment data, the virtual training scenario is dynamically adjusted to generate scenario adjustment parameter data; in the process of adjusting the virtual training scenario, the second physiological signal data and the second EEG signal data in the training process are collected and combined with the scenario adjustment parameter data to generate scenario impact data; the virtual training scenario is adjusted in real time according to the scenario impact data, and the training effect of the pilot is evaluated to generate training effect data; Based on the training effect data and the pilots' historical data, the reinforcement learning algorithm is used to generate personalized training analysis data. Based on the personalized training analysis data, personalized training plan data is generated, and in subsequent training, the personalized training effect of the pilots is optimized by adjusting the training content, the difficulty of the virtual training scenarios, and the feedback intensity.

[0005] Preferably, in the step of fusing all feature data, fusion is performed through an adaptive multimodal fusion algorithm, which calculates feature correlation and relative contribution value based on the change trend of each feature data, and uses a weighted average method to dynamically adjust the weight coefficient of each feature data to generate comprehensive feature data.

[0006] Preferably, the interactive calculation of multimodal features includes dynamic feature interaction analysis based on an interaction matrix, wherein the construction of the interaction matrix is ​​achieved by calculating a matrix of mutual correlation coefficients between different feature data, wherein the mutual correlation coefficients reflect the degree of association between different feature data, and the mutual correlation coefficients are dynamically updated according to the rate of change of each feature data in the time series.

[0007] Preferably, in the step of performing time-dependence analysis on the interaction feature data, a recursive neural network is used to perform time series modeling on the interaction feature data, wherein the input of the recursive neural network is the time series of the interaction feature data, and the output is the predicted future pressure trend. The recursive neural network updates the weights of the recursive neural network through forward propagation and error back propagation algorithms, and at the same time combines the attention mechanism to focus on feature changes related to the current situation, thereby generating comprehensive pressure trend data.

[0008] Preferably, the step of calculating the psychological stress index includes taking the interactive feature data and the comprehensive stress trend data as inputs of a multi-task deep learning model, wherein the shared layer of the multi-task deep learning model extracts global features through a convolutional neural network, and the task-specific layer performs calculations for specific tasks according to the type of features to generate a physiological state score and a cognitive load index, and generates a psychological stress index through the following calculation formula:

[0009] in, It represents the psychological stress index; Indicates physiological status score; represents the cognitive load index; and are dynamic weight coefficients, which are used to balance the contributions of the physiological state score and the cognitive load index, respectively. The dynamic weight coefficients are determined by a dynamic adjustment strategy according to the changes and importance of the feature data.

[0010] Preferably, in the step of dynamically adjusting the virtual training scenario based on the training intensity adjustment data, the changes in the virtual training scenario are regulated by controlling the light intensity, background sound effects, frequency and complexity of emergencies in the virtual training scenario, and the degree of emotional intervention on the pilot in the scenario.

[0011] Preferably, in the step of adjusting the virtual training scenario in real time, the virtual training scenario is feedback-controlled based on the scenario impact data through a closed-loop control system, and the closed-loop control system adopts a deep reinforcement learning algorithm to dynamically adjust the parameters of the virtual training scenario according to the scenario changes and feedback results, and the deep reinforcement learning algorithm optimizes the parameters in the policy function through policy gradient to maximize the scenario fitness.

[0012] Preferably, in the step of evaluating the training effect of the pilot, a regression model is used to predict the pilot's stress response, and the input of the regression model includes the second physiological signal data and the second EEG signal data. By analyzing the amplitude of the physiological signal change and the EEG signal frequency band characteristics, the reaction time, stress state score and decision accuracy score are generated.

[0013] Preferably, in the step of generating personalized training analysis data using a reinforcement learning algorithm, the reinforcement learning algorithm adopts a policy gradient optimization algorithm to maximize the long-term reward in each training by updating the weight parameters in the policy function. The policy gradient optimization algorithm achieves continuous optimization of the personalized training plan by associating feedback from the training scenario with long-term rewards.

[0014] A system for implementing the flight training control method, comprising: A multimodal data acquisition module, used for acquiring the first physiological signal data, the second EEG signal data and the visual signal data of the pilot, and performing preprocessing and feature extraction on the acquired data to generate corresponding feature data; A data fusion module, used for fusing the feature data of the first physiological signal data, the second EEG signal data and the visual signal data to generate comprehensive feature data; The feature interaction calculation module is used to perform interactive calculation of multi-modal features on the comprehensive feature data to generate interactive feature data, and to perform time-dependent analysis on the interactive feature data in combination with historical pressure change data to generate comprehensive pressure trend data; A psychological stress index calculation module, used to calculate the psychological stress index based on the interactive feature data and the comprehensive stress trend data, and generate real-time feedback parameter data and training intensity adjustment data; A virtual training scenario adjustment module is used to dynamically adjust the virtual training scenario based on the training intensity adjustment data to generate scenario adjustment parameter data; during the virtual training scenario adjustment process, the second physiological signal data and the second EEG signal data during the training process are collected and combined with the scenario adjustment parameter data to generate scenario impact data; A real-time scenario adjustment module is used to adjust the virtual training scenario in real time according to the scenario impact data, evaluate the pilot's training effect, and generate training effect data; The personalized optimization module is used to generate personalized training analysis data based on the training effect data and the pilot's historical data using a reinforcement learning algorithm, and to generate personalized training plan data based on the personalized training analysis data. It optimizes the pilot's personalized training effect by adjusting the training content, the difficulty of the virtual training scenario, and the feedback intensity.

[0015] Compared with the prior art, the advantages and beneficial effects of the present invention are: Through the multimodal data fusion, adaptive feature analysis and reinforcement learning control technology means of the present invention, a comprehensive dynamic control effect of the flight training process is achieved, solving the problems of slow heart rate detection and inflexible training scenario adjustment in the prior art; The present invention integrates multiple signal data, performs dynamic analysis based on time series, and combines reinforcement learning to achieve situational control, which greatly improves the real-time adjustment capability of training situations and the personalized training effect. By combining recursive neural networks with multi-task deep learning models, the comprehensive assessment of pilots’ physiological state and cognitive load is optimized, significantly improving the ability to accurately judge and provide feedback on training effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is a structural diagram of the interactive calculation of multimodal features in the present invention; Figure 3 A structural diagram of the virtual training scenario adjustment in the present invention; Figure 4 A structural diagram of the reinforcement learning algorithm in the present invention for optimizing personalized training; Figure 5 This is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0017] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0018] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0019] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0020] like Figure 1 As shown, a flight training control method comprises the following steps: Collect the pilot's first physiological signal data, second EEG signal data and visual signal data, and perform preprocessing and feature extraction on the collected data to generate corresponding feature data; fuse all feature data to generate comprehensive feature data; Preferably, in the step of fusing all feature data, fusion is performed through an adaptive multimodal fusion algorithm, which calculates feature correlation and relative contribution value based on the change trend of each feature data, and uses a weighted average method to dynamically adjust the weight coefficient of each feature data to generate comprehensive feature data.

[0021] In the context of flight training control, especially in the field of civil aviation, it is crucial to ensure that the pilot's physiological and psychological state is stable during the training process. In order to achieve this goal, the present invention is based on the collection and fusion of multimodal data, and evaluates the pilot's state by analyzing the pilot's physiological signals, EEG signals and visual signals, and provides a basis for control. Specifically, the present invention adopts an adaptive multimodal fusion algorithm to achieve data fusion, which is the key to achieving flight training control. In the present invention, the focus of data collection is on the pilot's first physiological signal data, second EEG signal data and visual signal data. These data include physiological characteristics such as heart rate, respiration, skin conductance, EEG activity signals, and visual characteristics such as eye movement trajectories and pupil changes obtained by high-resolution cameras. The principle of the present invention is to perform non-invasive measurements on pilots through a variety of sensors to obtain multimodal data. All collected signals will undergo a series of preprocessing processes after input, such as signal filtering, noise removal, data synchronization, etc., to ensure the accuracy and consistency of the data. These preprocessing steps are well-known technologies in the art, so only the core needs to be mentioned. Feature data is extracted from the preprocessed signals, and these feature data are an important basis for subsequent analysis. For feature extraction of physiological signals, such as heart rate variability analysis, skin conductance change curve, and breathing frequency amplitude; for EEG signals, conventional frequency domain analysis methods are used to extract power spectrum features of different frequency bands, such as Wave, wave and Waves, etc.; the feature extraction of visual signals focuses on pupil dilation, eye movement speed and fixation time. These feature data can reflect the physiological and psychological state of the pilot and provide a basis for further data fusion and interactive analysis.

[0022] After data preprocessing and feature extraction, an important innovation in the present invention is the application of an adaptive multimodal fusion algorithm. The core of the algorithm is to adaptively adjust the weight of each feature according to the changing trend of different feature data to achieve dynamic fusion of feature data. In flight training control, the physiological and psychological state of the pilot will be affected by many aspects, and the correlation and importance between various feature data will change at any time. Therefore, relying only on a single feature data may be misleading and ignore the importance of other data. The adaptive multimodal fusion algorithm realizes comprehensive evaluation of data by performing correlation calculation and trend analysis on each feature data. Specifically, the adaptive multimodal fusion algorithm is based on the following steps: First, the algorithm calculates the correlation coefficient between each feature data, such as the correlation between physiological signals and EEG signals, and the cross-correlation between visual signals and EEG signals. The correlation can be calculated by methods such as cross-correlation analysis or Pearson correlation coefficient. Then, based on the feature correlation, the algorithm dynamically calculates the relative contribution of each feature data. This contribution reflects the importance of each feature data to the overall state evaluation at the current moment. Finally, a comprehensive feature data is generated by weighted averaging the contribution of each feature data. The weighted average process is expressed by the following formula:

[0023] in, is the comprehensive feature data, For the feature data, is a weight coefficient that is dynamically adjusted according to the contribution of the feature. Through this weighted average method, the algorithm can automatically adjust the influence of each feature when the pilot's state changes. During flight training, by collecting multiple signals and using an adaptive multimodal fusion algorithm to generate comprehensive feature data, it helps to accurately reflect the pilot's comprehensive state. This state information can not only be provided to the pilot himself in real time as feedback for self-regulation, but also provide a basis for the training system to trigger necessary situational adjustments and real-time feedback mechanisms.

[0024] By integrating multimodal feature data, the psychological and physiological state of pilots can be accurately and comprehensively assessed to avoid misjudgment caused by single data analysis. For example, when a pilot's heart rate rises in a high-intensity situation, the EEG signal The wave decreases significantly, while the dilation amplitude of the pupil increases. The combination of these three can indicate that the pilot is in a highly stressed state, thereby triggering the system to reduce the intensity of the training situation. Compared with the analysis that relies solely on a single physiological signal, this multimodal fusion method can more reliably reflect the true state of the pilot and reduce the risk of misjudgment and missed judgment. Through the adaptive multimodal fusion algorithm, the present invention can flexibly and in real time adjust the weights of each feature data, so that the training control system can make the comprehensive state assessment of the pilot more comprehensive and accurate. Especially in the field of civil aviation, the training of pilots must take into account multiple influencing factors to ensure stability in a highly stressful environment. Therefore, the effect of the present invention significantly improves the training efficiency of pilots, reduces unnecessary stress reactions and misjudgments, and thus ensures the safety and effectiveness of training.

[0025] Perform interactive calculation of multimodal features on the comprehensive feature data to generate interactive feature data; perform time-dependent analysis on the interactive feature data in combination with historical pressure change data to generate comprehensive pressure trend data; calculate the psychological stress index based on the interactive feature data and comprehensive pressure trend data, and generate real-time feedback parameter data and training intensity adjustment data; The psychological stress management of pilots is crucial to ensure flight safety and training effects. To this end, the core of the present invention is to generate a psychological stress index through interactive calculation of multimodal features and time-dependent analysis, so as to provide a control basis for the training system, and then optimize the training scenario and feedback mechanism. First of all, the "interactive calculation of multimodal features of comprehensive feature data" in the present invention refers to the realization of feature interaction matrix analysis. The core of this analysis process is to reveal the correlation between each feature data to better reflect the overall psychological state of the pilot in different situations. Specifically, the comprehensive feature data is a data set fused from multiple signal sources (such as physiological signals, EEG signals, and visual signals), and each signal represents the state of the pilot in a specific dimension. Through the interactive calculation of multimodal features, a feature interaction matrix can be constructed, and the element values ​​of the matrix are expressed by cross-correlation analysis or Pearson correlation coefficient to represent the correlation strength between different feature data. This interactive analysis method can reveal the dynamic correlation between physiological features, EEG activities, and visual features, thereby generating "interactive feature data" and laying the foundation for subsequent time-dependent analysis.

[0026] For the step of "combining historical pressure change data to conduct time dependency analysis on interactive feature data", the present invention uses a recursive neural network (RNN) combined with an attention mechanism to model time series. The core of this principle is to capture long-term and short-term dependencies in time series through recursive neural networks, while the attention mechanism further enhances the flexibility of the model, enabling it to focus on feature data that is more important to the current situation. Specifically, in the time dependency analysis process, the input of the recursive neural network is a time series of interactive feature data, which reflects the characteristic changes of the pilot at different time points, and the output is the predicted future pressure change trend. The weight parameters of the network are continuously adjusted through the error back propagation algorithm, so that the model can better capture the changing patterns in the interactive feature data. At the same time, the introduction of the attention mechanism enables the model to dynamically choose to focus on the features that are more relevant at the current moment when predicting future pressure trends. For example, when the pilot is in a high stress state, he may pay more attention to changes in heart rate and EEG. This analysis process generates "comprehensive stress trend data" and further provides a basis for the calculation of the psychological stress index.

[0027] Calculating the psychological stress index based on "interactive feature data" and "comprehensive pressure trend data" is a key link in measuring the pilot's current psychological state in the present invention. In the specific implementation process, the scheme adopts a multi-task deep learning model to comprehensively process different input data. The shared layer of the model extracts global features through a convolutional neural network, and the task-specific layer calculates the respective scores according to different feature types, such as "physiological state score" and "cognitive load index". The weight coefficient is determined by a dynamic adjustment strategy based on the changes and importance of different feature data. The final psychological stress index reflects the pilot's overall psychological state in the current situation, and provides data support for training intensity adjustment and feedback.

[0028] By calculating the psychological stress index, the present invention generates "real-time feedback parameter data" and "training intensity adjustment data". The real-time feedback parameter data is used for the regulation of instant feedback during the training process. For example, when a pilot shows a high stress index, audio or visual prompts are used to help him adjust his state, while the training intensity adjustment data is used for more macroscopic training situation regulation. In specific applications, the training system can dynamically adjust the complexity of the situation, the intensity of the training content, or the frequency of sudden situations based on these data, thereby optimizing the pilot's training experience and effect.

[0029] In summary, the present invention accurately evaluates the psychological stress of pilots through interactive calculation and time-dependent analysis of multimodal features, and generates feedback and control data by calculating the psychological stress index. This comprehensive method not only enhances the accuracy of flight training control, but also improves the adaptability of pilots in stressful environments. Especially in the field of civil aviation, through this control system, the psychological stress response of pilots in high-intensity training or actual combat operations can be effectively reduced, thereby improving the safety and effectiveness of training.

[0030] Preferably, Figure 2 As shown, the interactive calculation of multimodal features includes dynamic feature interaction analysis based on an interaction matrix. The construction of the interaction matrix is ​​achieved by calculating a mutual correlation coefficient matrix between different feature data. The mutual correlation coefficient reflects the degree of association between different feature data, and the mutual correlation coefficient is dynamically updated according to the change rate of each feature data in the time series.

[0031] The pilot's reaction in a high-stress situation is crucial. In order to accurately evaluate and regulate the pilot's psychological state during training, the present invention adopts interactive calculation of multimodal features, performs dynamic feature interactive analysis through an interactive matrix, and realizes comprehensive monitoring of the pilot's comprehensive state. The core idea of ​​the interactive matrix is ​​to reveal and quantify the correlation between different feature data, and provide a reliable decision-making basis for training regulation. In the present invention, "interactive calculation of multimodal features" aims to analyze the dynamic correlation between multiple physiological, EEG, and visual signals by constructing an interactive matrix, and capture the state changes of pilots in complex situations.

[0032] Specifically, the construction of the interaction matrix is ​​achieved by calculating the "inter-correlation coefficient matrix" between different feature data. In this process, the inter-correlation coefficient is used to quantify the degree of correlation between two sets of feature data. In the application of the present invention, each element value of the inter-correlation coefficient matrix represents the correlation strength between different feature data, such as the heart rate change of the pilot and the brain wave signal in the brain wave signal. The relationship between changes in wave frequency bands, the degree of coupling between pupil dilation and physiological skin conductance signals, etc. Through the calculation of the values ​​of these interaction matrix elements, it can be intuitively reflected how the correlation between various characteristic data changes under different physiological and psychological states of pilots. The calculation of the mutual correlation coefficient usually adopts the Pearson correlation coefficient method, which is a common method in the civil aviation field to measure the linear correlation between two variables. The value of the Pearson correlation coefficient is between -1 and 1. A positive value indicates a positive correlation, a negative value indicates a negative correlation, and a value close to 0 indicates no significant correlation. In flight training control, the dynamic calculation of this mutual correlation coefficient helps to identify the correlation changes between the physiological-EEG-visual characteristics of pilots in different stress situations in real time, providing a basis for subsequent control.

[0033] In order to further enhance the accuracy of the evaluation, the mutual correlation coefficient in the interaction matrix is ​​not static, but is dynamically updated according to the change rate of each feature data in the time series. During the flight training control process, the physiological and psychological state of the pilot will be affected by multiple factors, and the changes in the feature data may also show short-term fluctuations or long-term trends. Therefore, simple static analysis may not be able to fully reflect this complex change relationship. The introduction of the dynamic update mechanism dynamically adjusts the value of the mutual correlation coefficient by monitoring the change rate of the feature data in the time series. Specifically, the measurement of the change rate can be achieved by the difference method or sliding window calculation, so as to capture the rapid changes of the feature data in a short time interval. For example, when the pilot enters a high stress state, the change rate of the skin conductance signal may increase significantly, and this change will dynamically adjust the correlation weight between the skin conductance signal and the EEG signal.

[0034] This dynamic update mechanism plays an important role in the present invention, enabling the interactive matrix to reflect the pilot's state changes in real time and improve the accuracy of the overall evaluation. Through this interactive calculation method, a complete interactive feature data set can be formed, covering the dynamic associations between different features. In the actual application of flight training control, these interactive feature data can help the training system to more clearly understand the changes in various physiological and psychological indicators of pilots during training, so as to adopt appropriate training situation adjustments and feedback mechanisms. For example, in a high-intensity simulation training task, if the system finds through interactive matrix analysis that the positive correlation between the pilot's heart rate and EEG signal is significantly enhanced, and the correlation between the pupil dilation signal and the skin conductance signal is weakened, the system can judge that the pilot may be in a state of over-concentration caused by stress. At this time, the system can appropriately reduce the difficulty of the simulation task or prompt the pilot to relax through real-time feedback.

[0035] In general, through dynamic feature interaction analysis based on the interaction matrix, the present invention can capture the correlation changes of pilots in multimodal feature data in real time, especially the subtle changes in complex situations and high stress states. Compared with the traditional static analysis method, dynamic feature interaction analysis can more comprehensively and accurately evaluate the pilot's status, enhance the effect of training regulation, and ensure the psychological and physiological stability of the pilot during training. This method provides a more scientific regulation basis for flight training and improves the safety and effectiveness of training.

[0036] Preferably, in the step of performing time-dependence analysis on the interaction feature data, a recursive neural network is used to perform time series modeling on the interaction feature data, wherein the input of the recursive neural network is the time series of the interaction feature data, and the output is the predicted future pressure trend. The recursive neural network updates the weights of the recursive neural network through forward propagation and error back propagation algorithms, and at the same time combines the attention mechanism to focus on feature changes related to the current situation, thereby generating comprehensive pressure trend data.

[0037] The changes in pilots' reactions and psychological states in complex and high-pressure environments are key factors in training regulation. In order to accurately capture the dynamic pressure changes of pilots during training, the present invention uses a recurrent neural network (RNN) to perform time series modeling of the pilot's comprehensive state by performing time-dependent analysis on the interactive feature data. Recurrent neural networks have a natural advantage in processing time series data and can effectively capture the long-term dependencies between data. Therefore, they are used in the present invention to predict the future pressure trend of pilots.

[0038] Specifically, the present invention first constructs a time series data set from the interactive feature data obtained through interactive calculation. The input of the recursive neural network is this time series data set, and the output of the network is the predicted value of the change trend of the pilot's psychological stress in the future. In practical applications, the interactive feature data not only includes the characteristics of the pilot's physiological signals, EEG signals and visual signals, but also includes the correlation between these characteristics. The multidimensionality of this data makes the modeling process of the time series more complicated, so LSTM (long short-term memory) units are used in the recursive neural network. LSTM units can effectively alleviate the gradient vanishing or explosion problems that may occur in traditional RNNs when dealing with long-term dependency problems. Through the LSTM unit, the model can better remember historical information that is crucial to the prediction of future pressure trends, while ignoring irrelevant or noise information.

[0039] When training a recursive neural network, the present invention uses forward propagation and error back propagation algorithms to update the weights of the model. Specifically, in the forward propagation stage, the input time series data is passed to each LSTM unit in turn, and the corresponding output and intermediate state are generated. In the error back propagation stage, the weights of the LSTM network are updated using the back propagation algorithm by calculating the error between the actual pressure trend and the model prediction result, thereby continuously optimizing the model's prediction ability. This process based on historical data and model optimization enables the recursive neural network to gradually learn and capture the laws of pressure changes of pilots in different situations, thereby improving the accuracy of future trend predictions.

[0040] In order to further improve the accuracy of prediction, the present invention also introduces an attention mechanism. The core idea of ​​the attention mechanism is to dynamically pay attention to the feature changes that are most relevant to the current situation when processing long time series. Specifically, the attention mechanism calculates the importance weight of each feature data at different time points, and focuses on selecting and amplifying the feature information that is most valuable for prediction. In the implementation process of the present invention, the attention mechanism will evaluate the importance of the interactive feature data at each moment, so as to input the key features into the subsequent processing unit of the recurrent neural network. For example, in a high-intensity simulated training scenario, the attention mechanism may focus on heart rate and EEG. The dynamic changes between waves are emphasized, while other relatively stable signal features are given lower weight.

[0041] By combining recursive neural networks with attention mechanisms, the present invention can generate more accurate "comprehensive stress trend data". These data not only reflect the pilot's state change trend in the current situation, but also predict the pilot's psychological stress fluctuations in the future. In the actual application of flight training control, the system can adjust the difficulty of the training situation in time according to these stress trend data, or provide feedback and reminders when the pilot's stress is about to reach a certain warning threshold, thereby effectively reducing the pilot's stress response.

[0042] In general, by using recursive neural networks to perform time-dependent analysis on interactive feature data, the present invention can dynamically predict the pilot's future stress trend, especially in complex flight training scenarios, capture the long-term dependencies of multi-dimensional features, and dynamically focus on the feature changes most relevant to the current situation in combination with the attention mechanism. Compared with traditional static evaluation methods, the present invention greatly improves the accuracy and flexibility of stress prediction, thereby enhancing the effect of training regulation. Through this control method based on time series modeling, the flight training system can better manage and guide the pilot's psychological state and improve the safety and effectiveness of training.

[0043] Preferably, the step of calculating the psychological stress index includes taking the interactive feature data and the comprehensive stress trend data as inputs of a multi-task deep learning model, wherein the shared layer of the multi-task deep learning model extracts global features through a convolutional neural network, and the task-specific layer performs calculations for specific tasks according to the type of features to generate a physiological state score and a cognitive load index, and generates a psychological stress index through the following calculation formula:

[0044] in, It represents the psychological stress index; Indicates physiological status score; represents the cognitive load index; and are dynamic weight coefficients, which are used to balance the contributions of the physiological state score and the cognitive load index, respectively. The dynamic weight coefficients are determined by a dynamic adjustment strategy according to the changes and importance of the feature data.

[0045] The psychological stress of pilots in high-intensity and complex situations directly affects the stability of their decision-making and operations. Therefore, accurately assessing the psychological stress state of pilots has become one of the core tasks of the flight training system. The present invention achieves this goal by calculating the psychological stress index. The specific steps include: using the interactive feature data and the comprehensive pressure trend data as input, and using a multi-task deep learning model for analysis and calculation to generate the pilot's psychological stress index.

[0046] The multi-task deep learning model is a key technology in the present invention, which is designed to process different types of feature data and tasks simultaneously. The input data of the present invention includes interactive feature data and comprehensive pressure trend data, which reflect the physiological, psychological and cognitive states of pilots in specific training scenarios. In order to comprehensively evaluate these states, the present invention designs a deep learning model structure with a shared layer and multiple task-specific layers. The shared layer mainly extracts global features through a convolutional neural network (CNN). The core of the convolutional neural network is that it can capture local features and patterns in the data through convolution kernels, and extract higher-order features layer by layer. In the application of the present invention, the convolutional neural network of the shared layer can automatically identify the complex associations between the pilot's physiological signals, EEG signals and visual signals from the input multimodal data, thereby generating comprehensive global features.

[0047] After the global features are extracted, they are passed to the task-specific layer for further analysis and processing based on the type of features and the different tasks. The design of the task-specific layer includes multiple sub-networks that specialize in processing specific tasks. These sub-networks are responsible for calculating the physiological state score and the cognitive load index. The physiological state score mainly quantifies the pilot's physiological stress level by analyzing the characteristics of his physiological signals (such as heart rate changes, breathing rhythm, and skin conductance); while the cognitive load index reflects the degree of cognitive burden based on the characteristics of the pilot's EEG signals and visual signals. For example, by analyzing wave and The power change of the wave is used to judge the pilot's concentration. The task-specific layer can share feature representations between different tasks through multi-task learning, while achieving independent evaluation of different states, thereby improving the overall evaluation ability and computational efficiency of the model.

[0048] After generating the physiological state score and cognitive load index, the present invention calculates the psychological stress index through the above formula. The setting of the dynamic weight coefficient is particularly important in the present invention, because the impact of physiological stress and cognitive load on the overall stress of pilots is different in different situations. To this end, the present invention designs a set of dynamic adjustment strategies to update the weight coefficients in real time according to the changes and importance of feature data. In the specific implementation process, the dynamic adjustment strategy will preset the initial weights through statistical analysis of historical data under different scenarios, and update it online according to the real-time feature data of the pilot. For example, when the pilot is facing an emergency simulation scenario, the system may pay more attention to physiological stress indicators (such as heart rate changes). At this time, the weight May be automatically adjusted to a higher value.

[0049] This method of calculating the psychological stress index based on a multi-task deep learning model has a significant effect in flight training control. By inputting multimodal data into the structure of the shared layer and the task-specific layer, the system can comprehensively identify and evaluate the pilot's physiological and cognitive state, and dynamically balance the impact of different states through a weight adjustment strategy. Compared with the traditional single signal analysis method, the present invention not only improves the accuracy of psychological stress assessment, but also can better adapt to complex training situations. In practical applications, the system can adjust the intensity of the training situation or provide feedback prompts in a timely manner according to the pilot's real-time psychological stress index, thereby effectively reducing the pilot's excessive stress response in high-pressure situations.

[0050] In summary, the present invention uses a multi-task deep learning model to generate a psychological stress index, achieving a comprehensive quantitative assessment of the pilot's physiological state and cognitive load. The convolutional neural network of the shared layer extracts global features, the task-specific layer processes the feature calculation of different tasks, and combined with a dynamic weight adjustment strategy, the system can more accurately measure and regulate the pilot's psychological stress, thereby improving the safety and effectiveness of flight training.

[0051] Based on the training intensity adjustment data, the virtual training scenario is dynamically adjusted to generate scenario adjustment parameter data; in the process of adjusting the virtual training scenario, the second physiological signal data and the second EEG signal data in the training process are collected and combined with the scenario adjustment parameter data to generate scenario impact data; the virtual training scenario is adjusted in real time according to the scenario impact data, and the training effect of the pilot is evaluated to generate training effect data; like Figure 3As shown, the dynamic adjustment of training scenarios plays a vital role in the pilots' stress adaptation and ability improvement. The present invention dynamically adjusts the virtual training scenarios based on "training intensity adjustment data" to adapt to the pilots' real-time psychological and physiological states, while providing effective situational feedback to optimize the training effect. First, the present invention uses training intensity adjustment data to achieve regulation of virtual training scenarios. The training intensity adjustment data is generated based on the pilot's psychological stress index, the complexity of the current situation, and the pilot's real-time feedback, and is used to guide the system to adjust the intensity and situational settings of virtual training at different time points. The adjustment content of the virtual training scenario includes multiple aspects such as situational lighting, background noise, frequency of emergencies, complexity of events, etc. By dynamically adjusting these situational parameters, it is ensured that the pilot is always in a moderately challenging environment during the training process.

[0052] When the virtual training scenario is dynamically adjusted, the system will further collect the second physiological signal data and the second EEG signal data during the training process. The second physiological signal data usually includes the pilot's heart rate, breathing rate, skin conductance and other characteristics during the training process, while the second EEG signal data reflects the pilot's brain electrical activity state in the current scenario (such as Wave, Wave, The system will collect the signal data and the situation adjustment parameter data generated during the situation adjustment process. These data record the specific operation content and adjustment intensity of each situation adjustment, such as the change in light intensity, the increase or decrease in background sound effects, and the time of occurrence of emergencies. By combining the second physiological signal data, the second EEG signal data, and the situation adjustment parameter data, the system can generate a complete "situation impact data" that can fully reflect the pilot's physiological and psychological reactions under specific situation adjustments.

[0053] Based on the situational impact data, the system will make real-time adjustments to the virtual training scenario. In this process, the system will adopt a closed-loop control strategy to continuously adjust the intensity and content of the scenario based on the pilot's response in a specific situation. In specific implementations, the situational impact data is used as feedback input, and the system will optimize the parameter configuration of the training scenario in real time based on the feedback information. For example, when the system detects that the pilot's heart rate rises sharply and his EEG rises sharply when responding to a complex emergency, the system will adjust the training scenario to the desired level. If the wave drops significantly, the frequency of similar emergencies can be appropriately reduced or their complexity can be reduced. Through this real-time situation adjustment, the system can effectively prevent pilots from being overly stressed and provide a more gradual training environment.

[0054] After completing the real-time adjustment of the training scenario, the system will evaluate the pilot's training effect and generate training effect data. The evaluation method of training effect data is based on the comprehensive analysis of physiological and EEG signals. By comparing the stress level, cognitive load and operation accuracy before and after training, the pilot's adaptation degree and training effect are judged. For example, by analyzing the pilot's heart rate variability and EEG at the end of training, the pilot's training effect can be evaluated. The wave recovery situation can measure the pilot's ability to recover when dealing with complex situations. The training effect data can not only serve as the basis for evaluating the results of this training, but also provide data support for the adjustment of subsequent training content, thereby realizing the gradual optimization of personalized training.

[0055] In general, the present invention establishes a closed-loop training control system by dynamically adjusting the virtual training scenario based on the training intensity adjustment data, combined with the real-time feedback of the scenario impact data. This system can flexibly adjust the training intensity and scenario configuration according to the pilot's real-time status and scenario feedback, thereby improving the safety and effectiveness of training. Compared with traditional static training methods, the dynamic adjustment and real-time feedback mechanism of the present invention is more precise, and can effectively avoid excessive stress reactions of pilots in complex training environments. At the same time, it continuously optimizes the training content by evaluating the training effect, helping pilots to better adapt and improve their abilities.

[0056] Preferably, in the step of dynamically adjusting the virtual training scenario based on the training intensity adjustment data, the changes in the virtual training scenario are regulated by controlling the light intensity, background sound effects, frequency and complexity of emergencies in the virtual training scenario, and the degree of emotional intervention on the pilot in the scenario.

[0057] The virtual training scenario is crucial for improving the pilot's coping ability and regulating stress response. In order to achieve dynamic management of the pilot's status, the present invention realizes dynamic adjustment of the virtual training scenario by controlling the light intensity, background sound effects, frequency and complexity of emergencies, and the degree of emotional intervention in the scenario. The principle of this adjustment method is to influence the pilot's physiological and psychological state through these core elements in the scenario, provide a more realistic and flexible training environment, and thus improve the effectiveness of training.

[0058] In specific implementation, the adjustment of virtual training scenarios mainly includes the following aspects: First, the control of light intensity is mainly used to simulate the pilot's adaptability under different lighting conditions. By adjusting the light intensity in the virtual training scenario, the pilot's visual perception under different lighting conditions such as day, dusk, and night can be simulated. This adjustment has a direct impact on the pilot's visual response and operational accuracy. For example, at dusk or night, the pilot's field of vision will be limited, and may require higher concentration and reliance on instruments. Through changes in light intensity, the training system can guide pilots to adapt to a variety of flight environments.

[0059] Secondly, the control of background sound effects is also an important part of adjusting the virtual training scenario. Background sound effects can simulate environmental noise, wind speed changes, alarm sounds, etc. during flight. By adjusting the intensity and frequency of these background sound effects, the system can test the pilot's concentration and reaction ability in a high-noise environment. For example, in certain emergency situations, a sudden high-pitched alarm can test the pilot's emergency response speed, while continuous low-frequency noise can test the pilot's psychological tolerance. Changes in background sound effects can be adjusted dynamically, thereby guiding pilots to learn to stay calm and respond correctly in a variety of noise environments.

[0060] The frequency and complexity control of emergencies is one of the core adjustments to simulate the pilot's ability to cope with high-pressure situations. During flight training, the frequency of emergencies can be dynamically adjusted according to the pilot's real-time status. For example, when the pilot shows a high level of stress, the system can temporarily reduce the frequency of emergencies or reduce their complexity to avoid excessive stress reactions of the pilot. On the other hand, when the pilot's response state is relatively stable, the system can gradually increase the frequency of emergencies and increase the complexity of the events to simulate a situation closer to a real flight mission. Through this step-by-step approach, the pilot's emergency response capabilities can be steadily improved. For example, emergencies can include failures in warning systems, electrical system failures, or sudden weather changes, while the increase in complexity can be reflected in the duration of the event, the scope of impact, and the requirements for multi-task response.

[0061] In addition, the adjustment of the degree of emotional intervention is a relatively innovative content, which aims to adjust the emotional state of pilots through appropriate psychological intervention methods. Emotional intervention is mainly carried out through the virtual coach's voice prompts, visual prompts or simulation of specific events. For example, when the system detects that the pilot's psychological stress index is high, the virtual coach can use voice to comfort the pilot, prompting him to take a deep breath or relax his mind; when the pilot's state tends to be stable, the virtual coach can use motivational language to guide the pilot to maintain an efficient operating state. This kind of emotional intervention adjustment can be flexibly controlled, and the intensity and method of intervention can be adjusted through real-time data feedback, thereby effectively reducing the pilot's psychological fluctuations and improving his emotional management ability.

[0062] Through comprehensive adjustments to light intensity, background sound effects, frequency and complexity of emergencies, and the degree of emotional intervention, the dynamic control of virtual training scenarios can more flexibly and accurately adapt to the pilots' real-time status and training needs. The effects of this dynamic adjustment are reflected in the following aspects: First, it enables pilots to receive comprehensive adaptive training in a variety of complex environments and improve their ability to cope with different flight conditions; second, through the two-way closed-loop regulation of scenarios and feedback, pilots can better adapt to changes in emergencies and improve the efficiency of emergency response; finally, through the guidance of emotional intervention, the pilots' emotional fluctuations can be alleviated in a timely manner, reducing excessive stress reactions during training. This multi-dimensional training scenario adjustment provides pilots with a flight training environment that is closer to reality, thereby comprehensively improving their psychological stability and operational accuracy in complex flight missions.

[0063] In summary, the present invention realizes all-round adjustment of virtual training scenarios through dynamic scenario control based on training intensity adjustment data, including precise control of light intensity, background sound effects, frequency and complexity of emergencies, emotional intervention, etc. This multi-dimensional adjustment strategy not only improves the flexibility and adaptability of training, but also optimizes the psychological state and operating skills of pilots by dynamically adjusting the scenario, providing a more scientific and comprehensive solution for civil aviation flight training.

[0064] Preferably, in the step of adjusting the virtual training scenario in real time, the virtual training scenario is feedback-controlled based on the scenario impact data through a closed-loop control system, and the closed-loop control system adopts a deep reinforcement learning algorithm to dynamically adjust the parameters of the virtual training scenario according to the scenario changes and feedback results, and the deep reinforcement learning algorithm optimizes the parameters in the policy function through policy gradient to maximize the scenario fitness.

[0065] The training system needs to be dynamically regulated according to the pilot's real-time status and feedback from the training scenario to improve the effectiveness and adaptability of the training. To this end, the present invention proposes a virtual training scenario feedback regulation method based on a closed-loop control system. The core is to achieve dynamic adjustment and optimization of the scenario through the application of a deep reinforcement learning algorithm. Specifically, in the process of real-time adjustment of the virtual training scenario, the system adopts a closed-loop control structure to perform feedback regulation based on the scenario impact data.

[0066] The principle of the closed-loop control system is to use the situational impact data as feedback input to adjust the relevant parameters of the training situation according to the changes in the pilot's physiological and EEG signals during the training process. The situational impact data reflects the pilot's physiological stress and cognitive load status in the current training situation. These data can include changes in the pilot's heart rate, EEG, and other factors. wave and The amplitude of the wave, the degree of pupil dilation and other characteristic information. By analyzing these data in real time, the closed-loop control system can dynamically determine whether the current situation setting is appropriate, and then make adjustment strategies. In order to achieve more efficient situation feedback control, the present invention introduces a deep reinforcement learning algorithm. The core of this algorithm is to dynamically optimize the training parameters according to the changes in the training situation and the feedback results.

[0067] The application of the deep reinforcement learning algorithm in the present invention is mainly achieved through policy gradient optimization, that is, during the training process, the system optimizes the parameters in the policy function according to the goal of maximizing the situational fitness (the interactive adaptation between the environment and the pilot). Specifically, the deep reinforcement learning algorithm iteratively updates the policy function through a policy gradient method (such as the PPO algorithm). After each adjustment of the virtual training scenario, the system will evaluate the effect of the adjustment, for example, by calculating the pilot's stress index, operation accuracy and other indicators in the current scenario to measure the rationality of the situation adjustment. Then, the system uses these indicators as feedback input to further optimize the situation parameters to improve the effect of the next adjustment. This step-by-step optimization process is achieved through the following mechanisms: In the present invention, the deep reinforcement learning algorithm constructs a state space based on the situational impact data. Each state point reflects the pilot's real-time state and the current training situation configuration, such as the fluctuation range of physiological characteristic data, the rate of change of EEG characteristics, the setting of situational parameters such as light and sound effects, etc. In each state, the system selects the optimal situation adjustment operation based on the strategy function. These adjustment operations include parameter settings for various training situations such as light, noise, frequency of emergencies, emotional intervention, etc. Through this strategy optimization, the system can ensure that the changes in the training situation and the pilot's fitness are always maintained at the optimal level.

[0068] The algorithm uses the policy gradient method to adjust the parameters of the policy function to maximize the situational fitness. In the specific implementation, the situational fitness is measured by the pilot's physiological response and psychological stress changes in the adjusted situation. For example, in a high-stress situation, the pilot's heart rate fluctuates greatly and his EEG changes The wave is significantly reduced, and the situational fitness is low at this time. The system will improve the fitness by adjusting the light intensity or reducing the frequency of emergencies. Through the policy gradient method, the system can dynamically update the weight coefficient of the policy function according to the change of situational fitness after each adjustment, so that the system gradually learns to choose a more appropriate situation adjustment strategy.

[0069] This combination of closed-loop control and deep reinforcement learning enables the system to achieve adaptive situational adjustments during flight training. Compared with traditional static training methods, this dynamic control method has the following significant advantages: First, it can automatically adjust the intensity and content of training according to the pilot's real-time status and physiological feedback, avoiding excessive stress for pilots in high-pressure situations. Secondly, by continuously optimizing the situational adjustment strategy, the system can gradually improve the pilot's coping ability and help them adapt to complex flight situations more quickly. Finally, through the policy gradient optimization of deep reinforcement learning, the system can accumulate experience in continuous training and form a more stable and effective situational adjustment strategy to ensure that the effect of each training is maximized.

[0070] In summary, the present invention achieves precise dynamic control in the adjustment of virtual training scenarios by introducing closed-loop control and deep reinforcement learning algorithms. Through feedback control based on situational impact data, the system can flexibly adapt to the pilot's real-time status and training needs, and continuously optimize the intensity of training and situational configuration to maximize the adaptability of the situation. This intelligent control method not only improves the flexibility and accuracy of training, but also provides pilots with a more realistic and personalized training environment, thereby significantly improving the effectiveness and safety of flight training.

[0071] Preferably, in the step of evaluating the training effect of the pilot, a regression model is used to predict the pilot's stress response, and the input of the regression model includes the second physiological signal data and the second EEG signal data. By analyzing the amplitude of the physiological signal change and the EEG signal frequency band characteristics, the reaction time, stress state score and decision accuracy score are generated.

[0072] The evaluation of pilot training effects is crucial to confirming the achievement of training goals, improving training effects and safety. The present invention predicts the pilot's stress response by using a regression model to comprehensively evaluate the pilot's reaction and decision-making performance in different training scenarios. Specifically, the regression model in the present invention uses the second physiological signal data and the second EEG signal data as input, and generates key evaluation indicators by analyzing the change amplitude and frequency band characteristics of these signal data: reaction time, stress state score and decision accuracy score.

[0073] The core of the regression model in the present invention is to capture the physiological stress response by using the pilot's second physiological signal data, which includes indicators such as heart rate, respiratory rate, and skin conductance. For these physiological signals, the system will focus on analyzing their change amplitude and fluctuation frequency. For example, when the pilot is operating under high pressure, the heart rate fluctuation may increase significantly, and the respiratory rate becomes more rapid. These abnormalities in the amplitude of change can be used as key features of physiological stress. In addition, changes in skin conductance can also reflect the pilot's autonomic nervous system response under stress. Through these physiological signal characteristics, the regression model can extract and quantify indicators such as reaction time. Reaction time is an important evaluation indicator, which is mainly calculated by analyzing the delay in the pilot's physiological signal response to emergencies during operation. The regression model can accurately measure the pilot's reaction delay by detecting the peak of heart rate and respiratory rate, combined with the event trigger time.

[0074] In the analysis of the second EEG signal data, the regression model focuses on the extraction and analysis of different frequency band features, which include Wave, Wave, The power spectrum changes of waves, etc. The characteristics of the pilot's EEG signals in different situations can reflect their cognitive load and concentration level, for example, A decrease in waves generally indicates a higher level of alertness, while The increase of the wave means a higher cognitive load. By extracting the changes in these frequency bands, the regression model can further generate a stress state score. The generation process of the stress state score is based on the comprehensive analysis of physiological signals and EEG signals. That is, the model will normalize these signal data and perform multivariate regression based on the amplitude and trend of feature changes to obtain a comprehensive score reflecting the pilot's stress level.

[0075] The regression model of the present invention also evaluates the pilot's decision accuracy score by analyzing the frequency band changes of EEG signals and the stability of physiological signals. The decision accuracy score mainly measures whether the pilot's reaction judgment in the training situation is appropriate and whether he maintains a high level of operation accuracy. Specifically, the model will evaluate the pilot's decision accuracy score based on the change pattern of the EEG signal in a certain event situation, such as wave and The dynamic changes of the wave are combined with the execution effect of the operation command (such as the accuracy of key response, the time accuracy of operation, etc.) to calculate a decision score. Through the comprehensive analysis of these data, the model can accurately determine whether the pilot's decision under high stress is correct, and quantify the corresponding score.

[0076] In summary, by using a regression model to predict the pilot's stress response, the present invention can generate reaction time, stress state score and decision accuracy score based on the changing characteristics of physiological and EEG signals. In specific flight training control, these evaluation indicators can provide accurate feedback information for the training system and help training designers understand the pilot's coping ability and psychological state in different situations. This comprehensive evaluation method based on signal characteristics not only improves the accurate evaluation of training effects, but also helps pilots better understand and manage their own stress reactions and optimize their operational performance in high-pressure situations. Compared with traditional evaluation methods, this regression model method based on multimodal signal analysis is more scientific and objective, and provides important guarantees for the safety and effectiveness of flight training.

[0077] Based on the training effect data and the pilots' historical data, the reinforcement learning algorithm is used to generate personalized training analysis data. Based on the personalized training analysis data, personalized training plan data is generated, and in subsequent training, the personalized training effect of the pilots is optimized by adjusting the training content, the difficulty of the virtual training scenarios, and the feedback intensity.

[0078] Preferably, in the step of generating personalized training analysis data using a reinforcement learning algorithm, the reinforcement learning algorithm adopts a policy gradient optimization algorithm to maximize the long-term reward in each training by updating the weight parameters in the policy function. The policy gradient optimization algorithm achieves continuous optimization of the personalized training plan by associating feedback from the training scenario with long-term rewards.

[0079] The design of personalized training programs is crucial to improving the adaptability and operational efficiency of pilots. In response to this demand, the present invention generates personalized training analysis data by using a reinforcement learning algorithm, and further generates personalized training programs based on these analysis data. The core of this process is to dynamically adjust the training scenario through a policy gradient optimization algorithm to maximize the long-term training return of the pilot, thereby achieving continuous optimization of the personalized training effect.

[0080] The core of the present invention is to combine the pilot's training effect data with historical data and use the reinforcement learning algorithm to generate personalized training analysis data. The training effect data includes indicators such as the pilot's stress state, reaction time, and operation accuracy in different situations, while the historical data records the various state characteristics and adaptation curves shown by the pilot in previous training. By combining these data, the system can understand the performance differences of pilots in different training situations. The task of the reinforcement learning algorithm is to establish a personalized strategy through these data, so that in subsequent training, the system can dynamically adjust the training content and the difficulty of the virtual training situation to maximize the long-term adaptability of the pilot.

[0081] Specifically, the implementation of the reinforcement learning algorithm in the present invention is based on the policy gradient optimization algorithm. The policy gradient optimization algorithm is a commonly used reinforcement learning method. Its core is to maximize the long-term return in each training by adjusting the weight parameters in the policy function. The so-called long-term return means that during the entire training process, the system comprehensively measures the effect of each training adjustment based on the pilot's real-time performance and historical performance. For example, if the pilot performs well in a certain type of situation, the system will gradually increase the difficulty of this type of situation. If the pilot shows a continuous high stress response in certain situations, the system will reduce the frequency of similar situations or reduce their intensity. The policy gradient optimization algorithm can gradually improve the training plan through continuous learning of these feedback data to make it more in line with the personalized needs of pilots.

[0082] like Figure 4 As shown in Figure 1, an important step in the practical application of reinforcement learning algorithms is to link feedback from training scenarios with long-term rewards. Specifically, after each training session, the system will evaluate the adjustment effect of the current training scenario, including the pilot's performance and stress response under different training difficulties. Based on these feedback data, the algorithm will update the weight parameters of the policy function so that in future training, pilots can gradually adapt to more difficult scenarios. Through this feedback and reward association mechanism, reinforcement learning algorithms can dynamically optimize personalized training plans.

[0083] In the application of the present invention, the generation of personalized training analysis data is mainly reflected in the following aspects: First, the system will identify the pilot's adaptability and operating status in different training scenarios based on the pilot's real-time performance, thereby obtaining personalized analysis data. Next, based on these analysis data, the system generates personalized training program data. The core of personalized training program data is to adjust the training content, the difficulty of virtual training scenarios, and the feedback intensity according to the pilot's performance. For example, when the system detects that the pilot performs poorly in a certain type of scenario, it can appropriately reduce the difficulty and increase the frequency of feedback prompts to help the pilot improve his skills in a more supportive scenario; conversely, when the pilot shows strong adaptability, the difficulty can be gradually increased and the feedback can be reduced to improve the pilot's self-judgment and emergency handling capabilities.

[0084] Through this reinforcement learning-based policy optimization, the present invention can dynamically generate and update personalized training programs, thereby continuously optimizing the pilot's training effect. The main advantage of this method is that the system can continuously adjust the training program according to the pilot's real-time feedback and performance curve, so that the training content is more in line with the pilot's personalized needs. In addition, the introduction of the policy gradient optimization algorithm makes the system more flexible in adjusting the training difficulty and feedback intensity, and can be dynamically adjusted according to the pilot's learning curve, thereby maximizing the pilot's long-term adaptability and operational level.

[0085] In general, the present invention generates personalized training analysis data using a reinforcement learning algorithm, and further generates personalized training plans based on these data, providing a continuously optimized training framework for pilots. By combining the pilot's training effect data with historical data, the system can dynamically evaluate and adjust the training plan, and maximize the long-term return of training through a policy gradient optimization algorithm. The application of the present invention in the field of civil aviation can not only improve the pilot's adaptability and operating accuracy, but also help the pilot adapt to changing flight situations more quickly, thereby significantly improving the safety and effectiveness of training.

[0086] A system for implementing the flight training control method, comprising: The multimodal data acquisition module is used to collect the pilot's first physiological signal data, second EEG signal data and visual signal data, and preprocess and extract features from the collected data to generate corresponding feature data; the core hardware of this module includes physiological signal sensors (such as heart rate sensors, skin conductance sensors, etc.), EEG sensors (such as EEG acquisition devices) and high-resolution cameras. The physiological signal sensor is used to collect data such as the pilot's heart rate, breathing rate and skin conductance changes, the EEG sensor collects the pilot's brain wave signals, and the camera is used to obtain the pilot's visual state. After preprocessing (such as noise filtering and feature extraction), the collected multimodal signal data forms preliminary feature data.

[0087] The data fusion module is used to fuse the feature data of the first physiological signal data, the second EEG signal data and the visual signal data to generate comprehensive feature data; the module uses multi-channel data fusion hardware, such as a high-performance processor (such as GPU or FPGA), to synchronize and fuse the data of different signal sources. Based on the physiological, EEG and visual signal feature data generated after preprocessing, the module uses an adaptive multimodal fusion algorithm to comprehensively analyze the correlation and importance between the signals, thereby generating a more comprehensive comprehensive feature data.

[0088] The feature interaction calculation module is used to perform interactive calculations of multimodal features on the comprehensive feature data to generate interactive feature data, and to perform time-dependent analysis on the interactive feature data in combination with historical pressure change data to generate comprehensive pressure trend data; this module relies on a high-performance computing platform and uses a mutual correlation coefficient calculation chip or an integrated computing accelerator to process data. The comprehensive feature data is input into the feature interaction calculation module, and the mutual correlation coefficients are analyzed by constructing an interaction matrix between different features. At the hardware level, this module can dynamically update weights in real time, perform time-dependent analysis, and generate comprehensive pressure trend data.

[0089] The psychological stress index calculation module is used to calculate the psychological stress index based on the interactive feature data and the comprehensive stress trend data, and generate real-time feedback parameter data and training intensity adjustment data; the implementation of this module depends on the deep learning acceleration chip (such as TPU or dedicated neural network processor). By inputting the interactive feature data and the comprehensive stress trend data, the module uses a multi-task deep learning model to calculate the psychological stress index. The hardware supports real-time calculation of the feature data to generate real-time feedback parameters and training intensity adjustment data.

[0090] The virtual training scenario adjustment module is used to dynamically adjust the virtual training scenario based on the training intensity adjustment data and generate scenario adjustment parameter data; during the virtual training scenario adjustment process, the second physiological signal data and the second EEG signal data of the training process are collected and combined with the scenario adjustment parameter data to generate scenario impact data; the core hardware of this module includes a high-performance simulation platform and a VR display device, which is responsible for real-time control of changes in virtual scenarios. Based on the training intensity adjustment data, the simulation platform can dynamically adjust scenario factors such as light intensity, background sound effects, and emergencies to generate scenario adjustment parameters. During the adjustment process, the second physiological and EEG signal data continue to be collected and combined with the scenario parameters to form scenario impact data.

[0091] The real-time situation adjustment module is used to adjust the virtual training situation in real time according to the situation impact data, evaluate the pilot's training effect, and generate training effect data; the hardware basis of the real-time situation adjustment module is the closed-loop control system and feedback sensor array. The module analyzes the situation impact data in real time and uses deep reinforcement learning algorithms to optimize the adjustment strategy so that the virtual situation can continuously adapt according to the pilot's real-time response. The core hardware of the module includes an efficient feedback controller and an adaptive learning unit.

[0092] The personalized optimization module is used to generate personalized training analysis data based on the training effect data and the pilot's historical data using the reinforcement learning algorithm, and generate personalized training plan data based on the personalized training analysis data, and optimize the pilot's personalized training effect by adjusting the training content, the difficulty of the virtual training scenario and the feedback intensity. The personalized optimization module relies on the reinforcement learning accelerator and the data storage unit. The module takes the pilot's training effect data and historical data as input, uses the reinforcement learning algorithm to dynamically generate personalized training analysis data, and generates a personalized training plan based on this. Hardware support enables rapid strategy updates and analysis optimization.

[0093] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

[0094] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A flight training control method, characterized in that: The following steps are involved: Collect the pilot's first physiological signal data, second EEG signal data and visual signal data, and perform preprocessing and feature extraction on the collected data to generate corresponding feature data; fuse all feature data to generate comprehensive feature data; Perform interactive calculation of multimodal features on the comprehensive feature data to generate interactive feature data; perform time-dependent analysis on the interactive feature data in combination with historical pressure change data to generate comprehensive pressure trend data; calculate the psychological stress index based on the interactive feature data and comprehensive pressure trend data, and generate real-time feedback parameter data and training intensity adjustment data; Based on the training intensity adjustment data, the virtual training scenario is dynamically adjusted to generate scenario adjustment parameter data; During the adjustment of the virtual training scenario, the second physiological signal data and the second EEG signal data are collected during the training process, and combined with the scenario adjustment parameter data to generate scenario impact data; the virtual training scenario is adjusted in real time according to the scenario impact data, and the training effect of the pilot is evaluated to generate training effect data; Based on the training effect data and the pilots' historical data, the reinforcement learning algorithm is used to generate personalized training analysis data; Generate personalized training plan data based on personalized training analysis data, and in subsequent training, optimize the personalized training effect of pilots by adjusting the training content, difficulty of virtual training scenarios and feedback intensity.

2. The flight training control method according to claim 1, characterized in that: In the step of fusing all feature data, fusion is performed through an adaptive multimodal fusion algorithm, which calculates feature correlation and relative contribution value based on the change trend of each feature data, and dynamically adjusts the weight coefficient of each feature data using a weighted average method to generate comprehensive feature data.

3. The flight training control method according to claim 1, characterized in that: The interactive calculation of multimodal features includes dynamic feature interaction analysis based on an interaction matrix. The construction of the interaction matrix is ​​achieved by calculating the mutual correlation coefficient matrix between different feature data. The mutual correlation coefficient reflects the degree of association between different feature data, and the mutual correlation coefficient is dynamically updated according to the change rate of each feature data in the time series.

4. The flight training control method according to claim 1, characterized in that: In the step of performing time-dependency analysis on the interactive feature data, a recursive neural network is used to perform time series modeling on the interactive feature data. The input of the recursive neural network is the time series of the interactive feature data, and the output is the predicted future pressure trend. The recursive neural network updates the weights of the recursive neural network through forward propagation and error back propagation algorithms, and at the same time combines the attention mechanism to focus on feature changes related to the current situation, thereby generating comprehensive pressure trend data.

5. The flight training control method according to claim 1, characterized in that: The step of calculating the psychological stress index includes taking the interactive feature data and the comprehensive stress trend data as inputs of a multi-task deep learning model, wherein the shared layer of the multi-task deep learning model extracts global features through a convolutional neural network, and the task-specific layer performs calculations for specific tasks according to the type of features to generate a physiological state score and a cognitive load index, and generates a psychological stress index through the following calculation formula: in, It represents the psychological stress index; Indicates physiological status score; represents the cognitive load index; and are dynamic weight coefficients, which are used to balance the contributions of the physiological state score and the cognitive load index, respectively. The dynamic weight coefficients are determined by a dynamic adjustment strategy according to the changes and importance of the feature data.

6. The flight training control method according to claim 1, characterized in that: In the step of dynamically adjusting the virtual training scenario based on the training intensity adjustment data, the changes in the virtual training scenario are regulated by controlling the light intensity, background sound effects, frequency and complexity of emergencies in the virtual training scenario, and the degree of emotional intervention on the pilot in the scenario.

7. The flight training control method according to claim 1, characterized in that: In the step of adjusting the virtual training scenario in real time, the virtual training scenario is feedback-controlled based on the scenario impact data through a closed-loop control system. The closed-loop control system adopts a deep reinforcement learning algorithm to dynamically adjust the parameters of the virtual training scenario according to the scenario changes and feedback results. The deep reinforcement learning algorithm optimizes the parameters in the policy function through policy gradient to maximize the scenario fitness.

8. The flight training control method according to claim 1, characterized in that: In the step of evaluating the training effect of the pilot, a regression model is used to predict the pilot's stress response. The input of the regression model includes the second physiological signal data and the second EEG signal data. By analyzing the amplitude of the physiological signal change and the EEG signal frequency band characteristics, the reaction time, stress state score and decision accuracy score are generated.

9. The flight training control method according to claim 1, characterized in that: In the step of generating personalized training analysis data using a reinforcement learning algorithm, the reinforcement learning algorithm adopts a policy gradient optimization algorithm to maximize the long-term return in each training by updating the weight parameters in the policy function. The policy gradient optimization algorithm achieves continuous optimization of the personalized training plan by associating feedback from the training scenario with long-term returns.

10. A system for implementing the flight training control method according to any one of claims 1 to 9, characterized in that: include: A multimodal data acquisition module, used for acquiring the first physiological signal data, the second EEG signal data and the visual signal data of the pilot, and performing preprocessing and feature extraction on the acquired data to generate corresponding feature data; A data fusion module, used for fusing the feature data of the first physiological signal data, the second EEG signal data and the visual signal data to generate comprehensive feature data; The feature interaction calculation module is used to perform interactive calculation of multi-modal features on the comprehensive feature data to generate interactive feature data, and to perform time-dependent analysis on the interactive feature data in combination with historical pressure change data to generate comprehensive pressure trend data; A psychological stress index calculation module, used to calculate the psychological stress index based on the interactive feature data and the comprehensive stress trend data, and generate real-time feedback parameter data and training intensity adjustment data; A virtual training scenario adjustment module, used to dynamically adjust the virtual training scenario based on the training intensity adjustment data and generate scenario adjustment parameter data; During the virtual training scenario adjustment process, the second physiological signal data and the second EEG signal data during the training process are collected and combined with the scenario adjustment parameter data to generate scenario impact data; A real-time scenario adjustment module is used to adjust the virtual training scenario in real time according to the scenario impact data, evaluate the pilot's training effect, and generate training effect data; The personalized optimization module is used to generate personalized training analysis data based on the training effect data and the pilot's historical data using a reinforcement learning algorithm, and to generate personalized training plan data based on the personalized training analysis data. It optimizes the pilot's personalized training effect by adjusting the training content, the difficulty of the virtual training scenario, and the feedback intensity.

Images