Flight simulation training auxiliary system and method based on intelligent co-pilot

Abstract

The invention discloses a flight simulation training auxiliary system and method based on intelligent co-pilot, and the system comprises a data collection module which is used for collecting flight data and carrying out the operation monitoring of a college; the voice interaction interface is used for multi-mode state analysis voice interaction; the student model module is used for skill level evaluation and learning style identification; the emotion perception module is used for flight experience identification and risk prediction analysis; the personalized training module is used for training process monitoring, training result analysis and personalized training adjustment; and the decision engine module is used for deeply reinforcing a learning model and assisting a strategy generator. And intelligent and personalized auxiliary support is provided for pilot training. Through real-time data analysis and an adaptive learning algorithm, functions and behaviors of a real copilot are simulated, and targeted guidance is provided for flight trainees. And a complete training ecosystem is formed. And the flexibility and expandability of the system are ensured, and each functional component can be independently optimized and upgraded.

Description

Technical Field

[0001] The present invention relates to the technical field of flight simulation training assistance systems, and in particular to a flight simulation training assistance system and method based on an intelligent co-pilot. Background Art

[0002] Currently, with the rapid development of the aviation industry, the demand for flight training is growing. Pilots are required to master comprehensive capabilities such as complex flight operations, emergency response, and multi-crew collaboration. Traditional flight simulation training systems are primarily based on preset flight scenarios and fixed procedures. They simulate the flight process through a physical simulation cockpit and a computer-generated visual environment. While they can reproduce the basic flight environment, they have significant shortcomings in the intelligent human-computer interaction, dynamic adaptability of training scenarios, and closed-loop management of student ability development. These deficiencies make them difficult to meet the high standards of modern flight training, as exemplified by the following core issues.

[0003] 1. Lack of primary pilot status awareness: Existing simulation systems rely heavily on mechanical monitoring of flight parameters (e.g., attitude, speed, and heading), lacking real-time, multi-dimensional awareness of the primary pilot's physiological state (e.g., eye movement trajectory, heart rate variability, muscle tension), cognitive load (e.g., working memory utilization, decision latency), and operational intent (e.g., tendency to correct abnormal inputs, multi-tasking priority judgment). For example, when a trainee experiences a distraction or stress response due to an unexpected situation, the system is unable to dynamically identify their ability shortcomings through biosignal and behavioral pattern analysis. Consequently, assistance strategies are limited to general prompts, making it difficult to provide precise intervention for individual weaknesses (e.g., insufficient instrument scanning frequency, confusing fault handling logic), severely limiting training efficiency.

[0004] 2. Insufficient adaptability of intelligent co-pilot systems: Current intelligent co-pilot modules are mostly implemented using rule engines or finite state machines. Their behavioral logic is rigidly based on pre-set scripts (e.g., standard call-out procedures, checklist execution), and they are unable to autonomously reason and adjust strategies based on real-time training context (e.g., sudden environmental changes, chain reactions caused by student misoperations). For example, during a simulated engine failure, the system only prompts restart steps according to a fixed time sequence, without being able to dynamically adjust prompt content and intervention timing based on the student's current operational stage (e.g., whether they have correctly identified warnings) or the state of cockpit resource allocation (e.g., whether the student is overly focused on a single instrument). This results in a mechanical training process and weakens the effectiveness of cultivating complex decision-making skills.

[0005] 3. Lack of realism in crew collaboration: Traditional systems rely primarily on one-way command transmission (e.g., button operation, simple voice commands) for human-machine interaction. They lack multimodal, natural interaction capabilities (e.g., semantic understanding, emotion recognition), making it difficult to create a highly realistic crew collaboration environment. For example, the co-pilot is unable to convey urgency through voice inflection or adaptively adjust their communication style based on trainees' operating habits (e.g., proactive questioning, collaborative confirmation, crisis warning). This makes it difficult for trainees to experience key collaborative scenarios such as dynamic division of labor and information cross-verification in a real cockpit, hindering the development of multi-crew collaborative capabilities.

[0006] 4. Single dimension of training assessment: The existing assessment system focuses too much on basic operational indicators (such as landing overload and route deviation), but lacks quantitative model support for high-level capabilities (such as operational fluency, risk prediction accuracy, and stress psychological stability). For example, the system can detect whether the trainee shuts down the faulty engine according to the process, but it cannot assess the efficiency of the remaining system redundancy during the handling process or the ability to predict subsequent chain risks. In addition, the assessment results are mostly presented in the form of static scores, lacking the ability to analyze the correlation between defects based on data mining (such as the sharp increase in the error rate of decision-making under specific weather conditions), resulting in a lack of targeted review guidance.

[0007] 5. Lack of personalized training adaptation mechanisms: Existing systems typically adopt a "one-size-fits-all" training model, failing to build a student's ability profile (such as experience level, cognitive style, and historical error pattern library). This makes it impossible to dynamically optimize training content based on individual differences. For example, when a junior trainee and a modified trainee face the same special situation, the system cannot differentiate between fault complexity (such as single-system failure versus multi-system coupled failure) or the intensity of co-pilot intervention (such as detailed guidance versus encouragement for independent decision-making). This leads to a mismatch between training content and students' current abilities, potentially leading to frustration due to excessive difficulty and wasting resources by repeating low-level training.

[0008] 6. Rigid emergency response assistance strategies: In special situation training, existing systems often adopt a bipolar assistance strategy of "full takeover" or "zero prompts," lacking a progressive guidance mechanism that conforms to the laws of cognitive learning. For example, during simulated wind shear recovery training, the system either completely takes over control and covers up the student's error, or allows the student to continue to deviate from the safety envelope until the end of the training. The system fails to design a gradient assistance strategy (such as providing principle prompts upon the first trigger, breaking down the operation steps when repeated errors occur, and forcing intervention before the critical crisis). This leads to students' over-reliance on the system or the inability to form a stable mental model of emergency response, which in turn weakens the goal of cultivating independent response capabilities.

[0009] 7. Summary of Industry Technical Bottlenecks: Although the international aviation training sector has experimented with introducing technologies such as virtual reality and machine learning, the aforementioned issues remain unresolved systematically. This stems from the inability of existing technical architectures to deeply integrate key technologies such as multi-source biosensing, cognitive computing, reinforcement learning, and dynamic difficulty adjustment. This results in gaps in the intelligent co-pilot system's real-time perception-decision-interaction-evaluation chain. Therefore, developing an intelligent assistance system that can accurately sense pilot status, autonomously generate dynamic training scenarios, simulate natural crew collaboration, and support personalized competency development has become an inevitable technological direction to overcome the limitations of traditional simulation training and meet modern aviation safety requirements. Flight training is a critical step in the development of civil and military pilots. Traditional flight training primarily relies on physical aircraft and human instructors, which is costly and resource-limited. With the advancement of computer technology, flight simulators have become a crucial tool for flight training, capable of simulating various flight scenarios and emergency situations in a safe environment.

[0010] In summary, there is at least one of the following technical problems:

[0011] Lack of awareness of the primary driver's real-time status and needs, making it difficult to provide targeted assistance;

[0012] Intelligent co-pilot systems usually use preset scripts or simple rules, which lack adaptability;

[0013] The interaction mode is single, making it difficult to simulate the real crew collaboration environment and communication mode;

[0014] The assessment system is too simple, making it difficult to provide personalized training plans for trainees of different levels;

[0015] Difficulty adapting to the needs of trainers with different experience levels and learning styles;

[0016] The auxiliary strategies in emergency situations lack gradient and rationality, which is not conducive to the cultivation of students' ability to deal with problems independently. Summary of the Invention

[0017] The main purpose of the present invention is to provide a flight simulation training assistance system based on an intelligent co-pilot and a method of using the same, so as to solve the technical problems in the prior art, such as the lack of perception of the real-time status and needs of the main pilot, making it difficult to provide targeted assistance; the intelligent co-pilot system usually adopts preset scripts or simple rules, which are not adaptable enough; the interaction method is single, making it difficult to simulate the real crew collaboration environment and communication mode; the evaluation system is too simple, making it difficult to provide personalized training plans for trainees of different levels; it is difficult to adapt to the needs of trainees with different experience levels and learning styles; the assistance strategy in emergency situations lacks gradient and rationality, which is not conducive to the cultivation of trainees' ability to independently deal with problems.

[0018] To achieve the above objectives, according to one aspect of the present invention, a flight simulation training assistance system based on an intelligent co-pilot is provided, comprising:

[0019] Data acquisition module, used for flight data collection and academy operation monitoring;

[0020] Voice interaction interface, used for multimodal state analysis voice interaction;

[0021] Learner model module for skill level assessment and learning style identification;

[0022] Emotion perception module, used for flight experience recognition and risk prediction analysis;

[0023] Personalized training module, used for training process monitoring, training result analysis and personalized training adjustment;

[0024] Decision engine module, used for deep reinforcement learning models and assisting policy generators.

[0025] Preferably, the data acquisition module is responsible for obtaining key data during the training process, and through multiple sensor networks and simulator API interfaces, it captures data in real time, including a set of flight parameters such as speed, altitude, attitude, and position. It also includes recording the trainee's operating behavior, including the joystick and throttle input mode, the interactive operation of the instrument panel, and the execution order and timing of the flight program.

[0026] Preferably, the acquisition module performs real-time preprocessing and noise filtering on the raw data through a signal processing algorithm, a Kalman filter and a wavelet transform technique. The acquisition module also adopts a data integrity check mechanism to ensure that the data is not damaged or lost during transmission through a cyclic redundancy check and a checksum algorithm.

[0027] Preferably, the student model module is responsible for constructing and maintaining a digital capability portrait of each student. The student model module is based on Bayesian statistics and hidden Markov models, and forms a multi-dimensional capability assessment by analyzing the student's historical training data. The student model module includes key dimensions such as technical operation capability, decision-making efficiency, emergency response, resource management and program execution, and assigns quantitative indicators and uncertainty intervals to each dimension.

[0028] Preferably, in terms of learning style identification, the student model module analyzes the student's reaction patterns and decision-making tendencies in different training scenarios through a recursive neural network, including micro-behavioral characteristics of risk preferences, hesitation patterns, and attention allocation habits.

[0029] Preferably, the student model adopts an online learning mechanism, which can continuously update and optimize itself as new training data accumulates. The updating process adopts an incremental learning algorithm, and the weight of the new data is dynamically adjusted according to the time decay function to reflect the student's latest ability status. At the same time, a self-calibration mechanism is adopted to regularly verify the accuracy of the model through standardized test tasks and make necessary parameter adjustments.

[0030] Preferably, the situational understanding module provides the system with in-depth flight context perception capabilities, accurately understands the semantic meaning and potential risks of the current flight status, and lays the foundation for intelligent decision-making. The situational understanding module is implemented through knowledge graphs and expert systems, temporal convolutional network analysis and risk prediction models. By integrating domain knowledge, real-time status analysis and future risk prediction, it achieves a dynamic, comprehensive and semantic understanding of the training scenario, providing key situational input for subsequent intelligent intervention and teaching strategies.

[0031] Preferably, the personalized training module builds a data-driven personalized training closed loop by deeply integrating intelligent modeling of learning curves, in-depth diagnosis of root causes of errors, and precise regulation of cognitive load; the voice interaction interface achieves a highly realistic co-pilot interaction experience by deeply integrating natural language processing, emotional state recognition, and crew collaboration logic.

[0032] Preferably, the decision engine module dynamically decides whether to intervene, the type and intensity of intervention based on the real-time status, balances the students' learning effect and flight safety, and avoids excessive intervention that damages the students' autonomous learning ability. The engine is responsible for integrating information from various modules of the simulator and generating intelligent auxiliary decisions. The real-time status includes flight parameters, student status and situational characteristics.

[0033] According to another aspect of the present invention, there is provided an assisting method for a flight simulation training assisting system based on an intelligent co-pilot, comprising:

[0034] The intelligent co-pilot system automatically adjusts its assistance level and method based on the main driver's behavior, performance and training stage, dynamically adjusting from full assistance to minimal intervention and adaptive learning;

[0035] Through speech recognition and natural language processing, natural communication and interaction with the primary pilot are achieved, simulating the communication mode in a real cabin environment and conducting multimodal voice interaction;

[0036] The system identifies the current flight phase, training objectives, and potential risks in real time, provides corresponding professional support and suggestions, and performs emotional perception;

[0037] Based on historical training data, learning curves and error pattern analysis, the training content and difficulty are dynamically customized for each trainee to provide personalized training;

[0038] Provide progressive emergency assistance. In an emergency, the system provides graded assistance based on the trainees' abilities, from prompts and suggestions to direct takeover when necessary, to cultivate the trainees' ability to handle problems independently.

[0039] The application of the technical solution of the present invention has the following technical effects:

[0040] From the above description, it can be seen that the above embodiments of the present invention achieve the following technical effects:

[0041] The Intelligent Co-pilot Flight Simulation Training Assistance System is an integrated training platform designed to provide intelligent, personalized support for pilot training. Based on artificial intelligence technology, the system uses real-time data analysis and adaptive learning algorithms to simulate the functions and behaviors of a real co-pilot, providing targeted guidance to student pilots. The system consists of six tightly integrated functional modules, centered around a decision-making engine, forming a complete training ecosystem. This modular design ensures both system flexibility and scalability, while also enabling independent optimization and upgrades of each functional component.

[0042] Intelligent decision-making is deeply enhanced. Existing systems mostly use static rules or shallow machine learning, which makes it difficult to dynamically balance safety and teaching needs. This patent achieves dual-time-scale precise control through a layered reinforcement learning architecture (high-level policy network + low-level execution network). Combined with Thompson Bayesian exploration of adaptive student characteristics, it ensures real-time performance (50Hz response) while ensuring that the intervention strategy is both targeted and robust.

[0043] Adaptive training intensity selection. Traditional systems rely on instructor experience to pre-set training paths and are unable to dynamically adapt to student status. This patented system pioneers a meta-learning-driven reward function and a three-dimensional intervention intensity matrix (student level × flight phase × mission complexity), enabling autonomous evolution of training strategies from "safety guarantee" to "performance breakthrough." Combined with differential cognitive load control and root-cause-driven stagnation response, training intensity is consistently precisely matched to the student's "zone of proximal development," accelerating skill acquisition compared to fixed-difficulty approaches.

[0044] Multi-module collaborative optimization: Existing systems operate in isolation, leading to cognitive overload and training disconnection. This patented technology uses prioritized experience replay (TD-error weighted sampling) to close the data loop between the decision engine and personalized training modules, prioritizing learning of safety events (such as ground proximity warnings). Simultaneously, the semantic context understanding engine (knowledge graph + temporal convolutional network) analyzes operational intent in real time, reducing trainees' cognitive load and forming an intelligent enhancement cycle of "perception-decision-execution-optimization." BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0046] Figure 1 A schematic structural diagram of a flight simulation training assistance system based on an intelligent co-pilot according to the present invention is shown;

[0047] Figure 2 Shown Figure 1 Workflow diagram of the flight simulation training assistance system based on intelligent co-pilot;

[0048] Figure 3 Shown Figure 1 The workflow diagram of the adaptive engine module of the flight simulation training assistance system based on intelligent co-pilot;

[0049] Figure 4 Shown Figure 1 Schematic diagram of the adaptive engine module of the intelligent co-pilot-based flight simulation training assistance system;

[0050] Figure 5 Shown Figure 1 The system timing workflow diagram of the flight simulation training auxiliary system based on intelligent co-pilot;

[0051] Figure 6 Shown Figure 1 Figure 2. Deployment edge-cloud hybrid architecture diagram of the intelligent co-pilot-based flight simulation training assistance system. DETAILED DESCRIPTION

[0052] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0053] like Figures 1 to 6 As shown, the embodiment of the present invention provides a flight simulation training assistance system based on an intelligent co-pilot, comprising: a data acquisition module for collecting flight data and performing academy operation monitoring; a voice interaction interface for multi-modal state analysis and voice interaction;

[0054] The student model module is used for skill level assessment and learning style identification; the emotion perception module is used for flight experience identification and risk prediction analysis; the personalized training module is used for training process monitoring, training result analysis and personalized training adjustment; the decision engine module is used for deep reinforcement learning models to assist the strategy generator.

[0055] In this embodiment, adaptive learning system: the intelligent co-pilot system can automatically adjust its assistance level and method according to the behavior, performance and training stage of the main pilot, and dynamically adjust from full assistance to minimal intervention. Multimodal voice interaction: through technologies such as voice recognition and natural language processing, natural communication and interaction with the main pilot can be achieved, simulating the communication mode in a real cabin environment. Situational awareness technology: the system can identify the current flight phase, training objectives and potential risks in real time, and provide corresponding professional support and suggestions. Personalized training path: based on historical training data, learning curves and error pattern analysis, the training content and difficulty are dynamically customized for each trainee. Progressive emergency assistance: in an emergency, the system provides graded assistance based on the trainee's ability, from prompts and suggestions to direct takeover when necessary, to cultivate the trainee's ability to deal with problems independently.

[0056] In this embodiment, Figure 1 The Intelligent Co-pilot Flight Simulation Training Assistance System is an integrated training platform designed to provide intelligent, personalized support for pilot training. Based on artificial intelligence technology, the system uses real-time data analysis and adaptive learning algorithms to simulate the functions and behaviors of a real co-pilot, providing targeted guidance to student pilots. The system consists of six tightly integrated functional modules, with a decision engine at its core, forming a complete training ecosystem. This modular design ensures both system flexibility and scalability, while also enabling independent optimization and upgrades of each component. The data acquisition module is responsible for acquiring all critical data during training. Through a multi-sensor network and simulator APIs, it captures a complete set of flight parameters, including speed, altitude, attitude, and position, in real time. An acquisition frequency exceeding 50 Hz ensures sufficient data for subsequent analysis. In addition to flight status parameters, it also records all student operational actions, including joystick and throttle input patterns, instrument panel interactions, and the order and timing of flight program execution. To ensure data quality, the acquisition module uses signal processing algorithms, including Kalman filters and wavelet transforms, to pre-process and filter noise from the raw data in real time. In addition, the module adopts a data integrity check mechanism to ensure that data is not damaged or lost during transmission through cyclic redundancy check and checksum algorithm.

[0057] The Student Model module is responsible for building and maintaining a digital capability profile for each student. Based on Bayesian statistics and hidden Markov models, this module analyzes students' historical training data to form a multi-dimensional capability assessment. The model covers key dimensions such as technical operational capability, decision-making efficiency, emergency response, resource management, and program execution, assigning quantitative indicators and uncertainty intervals to each dimension.

[0058] In terms of learning style identification, the system uses recursive neural networks to analyze students' response patterns and decision-making tendencies in different training scenarios, including micro-behavioral characteristics such as risk preferences, hesitation patterns, and attention allocation habits.

[0059] The student model utilizes an online learning mechanism, enabling continuous self-update and optimization as new training data accumulates. The update process employs an incremental learning algorithm, dynamically adjusting the weight of new data based on a time-decay function. This ensures that the model reflects the student's most recent state of ability while avoiding overcorrection due to single performance fluctuations. A self-calibration mechanism is also designed to regularly verify model accuracy through standardized testing tasks and make necessary parameter adjustments.

[0060] The situational understanding module provides the system with in-depth flight context awareness, accurately understanding the semantic meaning and potential risks of the current flight status, and laying the foundation for intelligent decision-making. This module is implemented through three core technologies:

[0061] Knowledge graph and expert system: Build and maintain a structured knowledge base containing massive aviation entities (flight procedures, systems, restrictions, etc.) and their complex semantic relationships, support rule-based rapid reasoning and fuzzy logic processing, and ensure that the system understands the position and meaning of operations in the standard procedure (SOP) framework.

[0062] Time Series Convolutional Network Analysis: Utilizes deep time series models to accurately identify detailed flight phases (12 phases) and operational modes from flight data streams. Powerful feature extraction automatically captures trends, cycles, and state transitions within the data. Furthermore, an autoencoder-based anomaly detection function identifies parameter combinations that deviate from normal patterns.

[0063] Risk Prediction Model: This model conducts forward-looking analysis, combining physical models and statistical learning (LSTM) to predict flight parameter changes over the next 5-20 seconds. By calculating the distance and closing speed between the predicted trajectory and a multi-dimensional safety envelope, it generates a standardized risk score (0-100) in real time, providing early warning of potential hazards.

[0064] This module achieves a dynamic, comprehensive and semantic understanding of training scenarios by integrating domain knowledge, real-time status analysis and future risk prediction, providing key contextual input for subsequent intelligent intervention and teaching strategies.

[0065] The decision engine system is the intelligent core of the flight training simulator. Its core objective is to dynamically determine whether to intervene, the type of intervention, and the intensity of intervention based on real-time status (including flight parameters, student status, and situational characteristics). This system aims to balance student learning outcomes (promoting growth) with flight safety, while avoiding excessive intervention that could compromise students' independent learning. The engine integrates information from various simulator modules to generate intelligent decision support.

[0066] Built on deep reinforcement learning and Bayesian decision theory, the system uses a dual network architecture to achieve stable policy learning. The input state is a high-dimensional vector that integrates flight parameters, student characteristics, and situational analysis; the output action is a set of discrete behaviors with different intervention intensities and assistance types. The core of the engine is a reward function that dynamically weighs three key factors: learning effect, safety factor, and degree of intervention. The weights are adjusted according to the training stage and student characteristics. The system adopts a Bayesian exploration strategy based on Thompson sampling, striking a balance between leveraging known knowledge and exploring unknown space to flexibly adapt to new students and scenarios.

[0067] To ensure real-time performance and stability, the engine employs a hierarchical decision-making architecture: a high-level policy network (updated approximately every 1-2 seconds) determines the overall intervention strategy and degree; a low-level execution network (running at a high frequency of 50Hz) translates high-level policies into specific auxiliary actions. A soft constraint mechanism connects these two layers, ensuring that low-level execution adheres to high-level intent while retaining flexibility to respond to unexpected situations.

[0068] The personalized training module involves the core adaptive engine of the intelligent co-pilot flight training system. It builds a data-driven personalized training closed loop by deeply integrating intelligent modeling of learning curves, in-depth diagnosis of root causes of errors, and precise control of cognitive load. The system uses a power-law function to establish a dynamic learning curve model for each core skill, detects skill plateaus in real time (the slope of 5 consecutive training sessions ≤ the threshold), combines Monte Carlo simulation to proactively optimize the training path, and locates the students' relative strengths / weaknesses through group benchmark comparison. A four-level error diagnosis system (technical / procedural / decision-making / communication) is designed to apply a hierarchical clustering algorithm to explore the common patterns behind surface errors (such as attention allocation defects under high load), trace the root causes of skill defects, knowledge deficiencies, or attention problems based on vectorized analysis, and generate targeted improvement strategies (such as procedural memory reinforcement or multi-task progressive training). The Cognitive Load Control Module integrates multiple physiological and operational indicators to construct a quantitative index and establishes a "fluid zone" control mechanism. This dynamically maintains training difficulty within the optimal challenge range (70-85% load utilization). When the load exceeds 90%, the difficulty is automatically reduced and reinforcements are provided. When the load is less than 60%, challenging tasks are injected, ensuring that training remains within the zone of proximal development. This module breaks through the limitations of traditional standardized training, implementing an intelligent closed-loop "assessment-diagnosis-optimization" system, improving training efficiency and targeting, enabling trainees to precisely address deficiencies and accelerate skill development at a reasonable intensity.

[0069] The voice interaction interface is the intelligent interaction core of the flight training simulator. By deeply integrating natural language processing, emotional state recognition and crew collaboration logic, it realizes a highly realistic co-pilot interaction experience. Its core functions are as follows:

[0070] Aviation-grade voice interaction features robust speech recognition capabilities, accurately understanding standard aviation terminology and non-standard expressions (such as "the value just mentioned") even in noisy cockpit environments. Based on a Transformer model and fine-tuned for the aviation field, it ensures accurate and efficient command intent recognition. Natural and clear speech synthesis provides context-appropriate feedback.

[0071] The multimodal emotion and stress perception system monitors students' status in real time, integrating and analyzing voice features (pitch, speaking rate, etc.) and physiological indicators (eye movements and optional heart rate) to accurately identify students' stress levels (normal, mild, moderate, and high). The system can capture stress trends (such as rapid increases) and adapt to individual differences through personalized calibration, providing a key basis for adjusting training difficulty and supporting strategies.

[0072] Intelligent Co-pilot Role Playing: The system dynamically simulates co-pilot role behaviors based on standard operating procedures (SOPs), flight phases, and scenarios. It implements five collaborative modes (full assistance, guidance, supervision, backup, and minimal intervention) and intelligently switches modes based on student capabilities, task complexity, and training objectives, providing appropriate support, from proactive guidance to safety monitoring.

[0073] This module uses the aforementioned technologies in synergy to enable trainees to obtain a better training experience and results through natural and situational interaction with the intelligent co-pilot.

[0074] In this embodiment, Figure 2 The system workflow shown below begins training by first loading student historical data (e.g., student A203's 50-hour training record). Using the Bayesian student model, a multidimensional capability profile is constructed (e.g., technical operation 76 / 100, decision-making efficiency 68 / 100). Training objectives are decomposed based on the skill map (e.g., crosswind landing accuracy), and the initial difficulty level (complexity level 4) is set using the cognitive load prediction model. The assistance intensity matrix is ​​then dynamically configured: intermediate student × landing stage × complexity 4 → baseline intervention intensity 0.6. The co-pilot is initialized to supervised mode (only prompts at key nodes).

[0075] Training execution phase

[0076] 1. Real-time data collection:

[0077] Multiple sensors capture flight parameters (such as airspeed 145 knots, crosswind 20 knots, throttle position 65%) at 50Hz, using wavelet noise reduction and CRC verification to ensure data integrity. The system also records student maneuver sequences (e.g., "flaps 30° → continuous right rudder correction").

[0078] 2. Situational understanding analysis:

[0079] The Temporal Convolutional Network (TCN) identifies the current phase as "final approach" (with 98% confidence), triggering the knowledge graph rule "crosswind > 15 knots, increase thrust and stabilize speed." The LSTM risk model predicts that the airspeed will fall below the safety threshold (138 knots in the next 10 seconds) and outputs a risk score of 72 / 100.

[0080] 3. Decision engine generates strategy:

[0081] The flight parameters (156 dimensions), student characteristics (low decision-making efficiency + visual learning), and situational characteristics (high risk) are integrated to generate a 300-dimensional state vector. The deep reinforcement learning policy network calculates the action probability distribution and ultimately outputs hierarchical instructions:

[0082] - High-level strategy: Provides guidance on airspeed correction

[0083] - Low-level execution: Voice "Recommendation to increase push by 5%"

[0084] 4. Multimodal interaction execution:

[0085] Voice interaction: 200ms delay broadcast command (optimized by aviation terminology BERT model)

[0086] Student response: Increase throttle to 70% and airspeed back up to 144 knots

[0087] Emergency Response

[0088] When a sudden drop in airspeed was detected (145 → 130 knots / 1 second), the risk prediction model score soared to 95 / 100:

[0089] 1) Bayesian confidence detection (0.63 < 0.7 threshold) triggers a conservative strategy

[0090] 2) The graded assistance mechanism responds according to the student level:

[0091] Intermediate student plan: Voice alert "Wind shear! Execute a go-around!"

[0092] -Compared to beginners: Added detailed operation instructions (such as "Keep the elevation angle at 10 degrees!")

[0093] 3) Error pattern library records "wind shear response delay of 2 seconds" in real time

[0094] 5. Evaluation feedback and personalized training stage

[0095] Based on a multi-dimensional assessment framework, a report is generated, showing the trainee's scores across different dimensions and the corresponding technical deficiencies. The root cause of the error is then analyzed, for example, attributing it to "lack of awareness of airspeed monitoring" and recommending specialized training scenario W-07. Personalized training planning is based on Monte Carlo path optimization: a wind shear training module is inserted (previously planned ahead of schedule); flow zone control: reducing crosswind intensity for the next mission (15 → ​​12 knots) to maintain cognitive load at 75%; and collaborative filtering recommendations: linking successful training examples from similar trainees (a combined crosswind and emergency handling scenario).

[0096] In this embodiment, the specific implementation of each module in the system is explained below.

[0097] The core adaptive capability of the intelligent co-pilot system is based on machine learning algorithms, which mainly include three key components: deep reinforcement learning implementation, Bayesian network model and auxiliary strength matrix. Its data flow, principle and timing diagram are as follows: Figure 3 、 4 , as shown in 5.

[0098] Next, the implementation of each module is described in more detail.

[0099] The decision engine system uses a dual network architecture to achieve intelligent decision-making. The policy network receives a 300-dimensional state vector (including 156-dimensional flight parameters, 84-dimensional student features, and 60-dimensional situational features), which is processed by an 8-layer fully connected neural network: the first 7 layers use the ReLU activation function for nonlinear transformation, and the number of neurons in each layer decreases successively (256→128→64→32→16→8→4). The final output layer generates a probability distribution of 50 auxiliary behaviors through the Softmax function. The target network is isomorphic to the policy network but uses an asynchronous update mechanism - the main network updates its parameters every 10 interactions, and the target network adjusts its parameters through a soft update strategy every 100 interactions.

[0100] θ target =0.99×θ target +0.01×θ policy .

[0101] Among them, θ target is the target network parameter, which is updated with lag; θ policy is the policy network parameter, which is dynamically updated; 0.99 is the target network update coefficient, and 0.01 is the policy network injection coefficient.

[0102] The system uses experience replay and priority sampling technology to optimize learning efficiency. The experience replay buffer stores the latest 100,000 interaction records. Each record contains state, action, reward, next state and termination flag. Each record stores a tuple (s t ,a t ,rt ,s t+1 ,done). Priority sampling based on time series difference error (TD-error) is adopted, and priority sampling is based on TD error δ=|r+γmax a′ Q target (s′,a′)-Q policy (s, a)|; where δ is the TD error, which means the difference between the target Q value and the predicted Q value r is the immediate reward, the reward value γ obtained in the current step is the discount factor, and the future reward attenuation coefficient Q target Target network Q value, delayed network output action value Q policy is the Q value of the policy network, the action value output by the real-time network, s is the environmental state, including the real-time state of the aircraft, the characteristics of the students, and the situational characteristics. a is the action including the actions taken and the auxiliary behaviors, the sampling probability is assigned, and the importance weight ω is used. i =(N·P(i)) -β (ω i is the importance weight, P(i) is the sampling probability, N is the buffer size, and the β importance weight coefficient increases linearly from 0.5 to 1) to correct the bias.

[0103] Reward Function

[0104] r t =α·Learning Gain(s t )-β·Safety Penalty(s t )-γ·Intervention Cost(a t ) where α is the learning benefit coefficient, initially 0.3, and increases with training β is the safety penalty coefficient, initially 0.7, and decays with training γ is the intervention cost coefficient, and is a fixed value weight coefficient α, β is dynamically adjusted through meta-learning: the safety weight β = 0.7 is dominant in the initial stage, and decays exponentially with the training time β·e -0.001t →β, the learning benefit weight α=0.3 gradually increases α·(1-e -0.002t )→α, where t is the number of training steps, achieving a gradual transition from safety assurance to training optimization. Learning effectiveness is calculated by the deviation of the student's current performance from the expected learning curve, with positive deviations receiving positive rewards. The safety factor is based on the distance between the current state and the safety margin, with closer distances resulting in greater penalties. Overintervention penalties are proportional to the frequency and intensity of system interventions.

[0105] The Bayesian network model provides the system with probabilistic reasoning and uncertainty handling capabilities. The network structure comprises three primary node clusters: student status, flight scenarios, and support requirements, totaling 48 nodes. Conditional dependencies are represented by directed edges. The core probabilistic reasoning is based on Bayes' theorem.

[0106] The system learns these conditional probability tables from historical data using the expectation maximization (EM) algorithm. Initialization requires approximately 10,000 labeled training data. The Bayesian network implements an online update mechanism and uses the exponentially weighted moving average method P updated (X|Y)=λ·P new (X|Y)+(1-λ)·P hist (X|Y),λ=0.8,P updated (X|Y) is the update probability; P new (X|Y) is the probability of new observation; P hist (X|Y) represents the historical probability; λ is the new data fusion weight, integrating newly observed data, giving it higher weight while preserving historical patterns. For regions with sparse data, the system uses the Dirichlet prior distribution Dir (α = 0.1) to smooth the conditional probability estimate. The system pays special attention to uncertainty handling, maintaining a confidence interval for each auxiliary decision. When the confidence level falls below a threshold (typically set at 0.7), the system automatically adopts a more conservative auxiliary strategy, prioritizing safety.

[0107] The auxiliary intensity matrix is ​​the core control mechanism of the system intervention strategy. It is designed as a three-dimensional matrix of 3×4×5, corresponding to the student level (beginner, intermediate, advanced), flight phase (takeoff, climb / cruise, descent, landing) and task complexity (from 1 to 5). The output value is the baseline intervention intensity m i j k ∈[0,1]. Each element in the matrix determines the baseline intervention intensity under a specific combination of conditions, ranging from 0 (no intervention) to 1 (complete takeover). The actual intervention intensity is further adjusted based on real-time risk assessment and student performance.

[0108] The matrix value is dynamically updated through two mechanisms: one is long-term adaptation based on the student's historical performance, which systematically analyzes the student's performance history in similar situations and gradually adjusts the corresponding matrix value. Among them, η is the long-term learning rate Original baseline strength New Benchmark Strength Perf expected Expected Points, Perf actual The system implements a short-term adjustment mechanism based on immediate feedback, monitoring the student's response to interventions and adjusting the strategy if the intervention is ineffective. The system implements an intelligent threshold mechanism, triggering intervention only when the student's performance deviation exceeds a specific threshold (correlated with the student's ability and the importance of the task), thus avoiding unnecessary interference. Furthermore, a gradual change strategy ensures a smooth transition in intervention intensity, preventing any disruption or safety risks that could arise from abrupt changes in intervention.

[0109] Multimodal interaction technology

[0110] It mainly includes three core components: natural language processing and speech recognition, sentiment analysis algorithm and crew resource management model.

[0111] The natural language processing and speech recognition system is based on a deep learning architecture and is optimized for the aviation environment.

[0112] The input audio undergoes multi-channel noise reduction (spectral subtraction + Wiener filtering) and feature extraction to produce an 80-dimensional Mel spectrum and Δ / ΔΔ features. The Δ feature refers to the first-order difference coefficient, essentially the rate of change of the spectrum between adjacent frames, capturing the dynamic characteristics of speech. The Δ / ΔΔ feature, a second-order difference, reflects the change in the rate of change. The static Mel spectrum reflects instantaneous acoustic characteristics, while the Δ / ΔΔ feature introduces temporal dynamic information, enabling the model to distinguish commands with similar pronunciation but different speech rates (such as "Climb" pronounced fast and slow), improving recognition accuracy in noisy environments. The input is then fed into an end-to-end Transformer-based model. Language representation is first constructed using a pre-trained BERT model, followed by domain-specific fine-tuning to improve recognition accuracy for aviation terminology. The system integrates multiple aviation terminology dictionaries, providing multiple pronunciation variants and contextual usage for each term, significantly improving recognition accuracy. To address the noise interference inherent in a real cockpit environment, the system employs multi-channel audio processing techniques and trains the model by adding real-world cockpit noise (including engine noise, communication interference, and airflow noise).

[0113] Contextual understanding is achieved through a long short-term memory (LSTM) network. The system maintains a conversation history buffer, storing the content and contextual features of the last 20 rounds of conversation. This enables the system to understand context-dependent instructions such as "Set it to the value just mentioned." The system also implements intent recognition, extracting operational intent and parameters from natural language and mapping them to specific system functions.

[0114] The sentiment analysis algorithm assesses students' emotional state and stress level using multimodal signal fusion technology. The system simultaneously analyzes voice features (such as pitch inflection, speaking rate, pause patterns, and volume) and physiological indicators (such as pupil dilation, blink rate, gaze movement patterns, and optional heart rate variability data obtained through eye tracking). These features are converted into a 160-dimensional emotion feature vector through a feature extraction network. This vector is then input into a support vector machine classifier to categorize stress levels into four levels: normal, mild stress, moderate stress, and high stress.

[0115] The system uses time-series pattern recognition technology to monitor stress trends. Through sequence analysis within a 10-second sliding window, the system not only detects absolute stress levels but also identifies dangerous patterns such as rapid increases in pressure or sustained high pressure. To accommodate individual differences, the system implements a personalized calibration process. During the initial training phase, baseline responses to various stress conditions are collected to construct a personalized stress model.

[0116] The crew resource management model implements the system's intelligent interaction capabilities as a co-pilot. The system dynamically adjusts the co-pilot role behavior according to standard operating procedures (SOPs) and automatically selects the appropriate interaction mode based on the current flight phase and situation. The system implements five different collaboration modes: full assistance mode (the system actively provides full support), guidance mode (the system provides guidance but the student makes the decision), supervision mode (the system mainly monitors and intervenes only when necessary), backup mode (the system acts as a safety net and intervenes only in dangerous situations) and minimum intervention mode (the system hardly intervenes and is mainly used for evaluation). Switching between these modes is automatically determined based on the student's ability level, task complexity and training objectives.

[0117] Situational awareness mechanism

[0118] The contextual awareness mechanism enables the system to understand complex flight environments and mission requirements, providing the necessary contextual information for decision-making. This mechanism mainly includes three core technical components: knowledge graph and expert system, time series convolutional network analysis, and risk prediction model.

[0119] The knowledge graph and expert system constitute the domain knowledge foundation of the system. The knowledge graph contains more than 20,000 nodes, covering various entities in the aviation field (such as flight procedures, system components, operating restrictions, emergency measures, etc.) and more than 78,000 semantic relationships between them. There are as many as 30 types of relationships between nodes, including "is a subroutine", "precedes", "requires", "influences", etc., forming a rich semantic network. The knowledge graph is extracted from flight manuals, training materials and expert interviews in a semi-automatic manner, and verified by aviation experts. The graph adopts a hierarchical structure, with the top-level concepts being relatively abstract (such as take-off procedures) and the bottom level containing specific details (such as "specific operations at the V1 decision point").

[0120] The reasoning engine based on the knowledge graph uses an improved Rete algorithm to implement forward chain reasoning. The algorithm optimizes the efficiency of rule matching by establishing pattern networks and connection networks, enabling the system to complete complex reasoning in milliseconds. To deal with uncertainties in the aviation field, the system integrates a fuzzy logic processing mechanism, allowing the rules to include fuzzy concepts (such as "approaching stall", "large deviation", etc.) and uncertain reasoning. The system maintains more than 3,000 expert rules verified by senior flight instructors, covering normal procedures, abnormal procedures and emergency procedures. These rules are expressed in a condition-action format, "IF the airspeed is lower than V2+10 knots AND the climb angle is greater than 15 degrees THEN there is a risk of stall."

[0121] The knowledge update mechanism allows the system to semi-automatically extract new knowledge points from training data. The system uses a transformer-based relation extraction model to analyze teacher feedback and training records, identify potential new rules and relationships, and then submit them to experts for review.

[0122] Temporal convolutional network analysis is responsible for identifying flight phases and operational modes from flight data. The network employs an 8-layer dilated causal convolutional architecture, with the dilation rate of each layer increasing by powers of 2 (1, 2, 4, 8, etc.). This allows the network to efficiently capture long-term dependencies, with a receptive field covering 60 seconds of flight data. This design enables the system to understand the meaning of current operations over a larger timescale, such as identifying complete approach procedures rather than just individual maneuvers.

[0123] The network's feature extraction capability enables it to automatically extract meaningful temporal features from raw flight parameters, such as trends, periodic patterns, and state transitions. Through self-supervised learning pre-training, the system learns temporal patterns from a large amount of unlabeled flight data, improving its generalization capabilities for various flight scenarios. In the flight phase recognition task, the system divides the flight into 12 phases.

[0124] The system's anomaly detection function is based on an autoencoder. It learns a low-dimensional representation of normal flight data and calculates reconstruction errors to identify anomalous patterns. When a parameter or combination of parameters deviates from the normal range, the system promptly detects and flags it as a potential anomaly. To optimize computational efficiency, the system employs quantization and sparse activation, enabling real-time processing on edge devices.

[0125] The risk prediction model identifies potential dangers in advance through forward-looking analysis. The system implements multi-step prediction capabilities and can predict flight parameter changes in the next 5-20 seconds based on current status and historical trends. The prediction uses a hybrid model approach, combined with a physical model (for short-term 0-5 seconds accurate prediction, x t+1:t+5 =f physics (u t ,wt );x t+1:t+5 is the predicted state, i.e. the state sequence for the next 5 seconds, f physics Represents the physical model of the flight dynamics equation, u t For control input, current operation instructions, etc. t for environmental disturbances, such as wind shear, turbulence, etc.) and statistical learning models (for long-term trend prediction, x t+6:t+20 =Seq2Seq-LSTM(x 0:t ,u 0:t )x t+6:t+20 To predict the state in the next 6-20 seconds, Seq2Seq-LSTM is used as the prediction model, encoder-decoder architecture, x 0:t is the historical state sequence, u 0:t The physical model is based on the flight dynamics equations, taking into account the current control inputs and environmental conditions; the statistical learning model uses a sequence-to-sequence LSTM network to learn complex nonlinear dynamics from historical data.

[0126] The risk scoring system calculates the risk index based on the distance between the predicted parameters and the safety envelope Among them, RiskScore is the comprehensive risk level, B i is the i-th safety boundary plane, v approach is the approach speed k is the number of boundaries x pred Prediction parameters represent the current or predicted state. The safety envelope consists of multiple parameter constraint surfaces. The system calculates the minimum distance and approach speed between the predicted trajectory and these constraint surfaces, comprehensively assessing the risk level. Risk scores are assigned using a standardized scale of 0-100, with multiple thresholds set to trigger different levels of warning.

[0127] Personalized training customization

[0128] The personalized training customization feature enables the system to optimize training paths based on student characteristics. This feature primarily includes four core technical components: learning curve analysis and stagnation detection; error pattern recognition and root cause analysis; cognitive load monitoring and dynamic regulation; and personalized training strategy generation.

[0129] The system establishes a separate learning curve model for each key skill and estimates parameter values ​​from historical training data using nonlinear regression methods. This personalized learning curve can accurately reflect the learning characteristics of different students.

[0130] Learning curve analysis using student historical datasets Perform feature extraction z = PCA-ANN (x); PCA-ANN is a hybrid dimensionality reduction, PCA linear dimensionality reduction, ANN nonlinear dimensionality reduction, z is the deep feature, that is, the feature after dimensionality reduction, x is the original dimension (where Contains characteristics such as training subjects and difficulty, i The system uses the power law learning curve model to quantify and predict the students’ skill development process. is the predicted score, a is the learning rate, b is the learning efficiency, and c is the initial ability, which indicates the change of student skill performance with the number of training times; parameters a, b, and c are obtained from the student historical data set through the Levenberg-Marquardt algorithm. Fitting historical data ( Contains characteristics such as training subjects and difficulty, i is the performance score of the current session). In the feature extraction stage, the original training record is reduced to an 8-dimensional feature vector by z = PCA-ANN(x). Stagnation detection is performed by calculating the recent performance slope, i.e. k is the stagnation criterion, i.e. the slope of recent performance t i is the training sequence number, the training number, y i Performance score, that is, the training result is the time mean After detecting performance stagnation, the system performs analysis and generates adjustment recommendations. If |k| < 0.05 for five consecutive training runs, a policy adjustment is triggered.

[0131] In the error pattern recognition and root cause analysis component, the system divides errors into four categories: technical, procedural, decision-making, and communication. Each category is further divided into multiple specific types, forming a detailed error classification system. After establishing the error classification system, the errors are vectorized and represented. Each error instance is coded as a 16-dimensional feature vector e = [type, context, frequency, severity] ∈ R 16 , where severity is scored by the expert rule base: Severity = (1, 0.3 × potential risk factor + 0.7 × actual consequence score);

[0132] The system uses a hierarchical clustering algorithm to identify common patterns in student errors. By quantifying error manifestations (including characteristics such as error type, occurrence context, frequency, and severity), the system can discover seemingly distinct but inherently similar error patterns. For example, the system may discover that multiple errors made by students during periods of high workload are actually due to attention allocation issues. This pattern recognition helps the system conduct root cause analysis, tracing the underlying causes of errors and distinguishing between different root causes such as skill deficiencies, knowledge gaps, and attention problems.

[0133] The specific error pattern recognition algorithm is implemented by using the improved Ward algorithm for multi-scale clustering, calculating the error vector similarity, initializing each error as a cluster, iteratively merging the cluster pairs that minimize the intra-class variance increment, stopping when the maximum cluster diameter is greater than the threshold d_max = 0.7, and then outputting K pattern clusters. The root cause analysis is achieved by constructing a three-layer neural network root cause classifier, from the input error example 16-dimensional feature vector to the output three-dimensional root cause weight vector [w_skill, w_knowledge, w_attention], satisfying ∑ w＝ 1; when w_attention>0.6, it is "attention allocation defect"; when w_knowledge>0.5 and related SOP steps, it is "insufficient procedural memory"; when w_skills<0.3 and the operating parameters have large deviations, it is "basic skills defect".

[0134] The dynamic cognitive load control function helps the system understand the student's mental burden and optimize training difficulty. The system comprehensively assesses cognitive load by constructing multiple indicators, including operation frequency F (the number of operations per unit time), operation complexity H (the diversity of operations based on information entropy), multitasking M (the number of tasks performed simultaneously), and response deviation R, such as reaction time and changes in operation accuracy. These indicators are weighted and combined to form a unified cognitive load index C_t = α·F + β·H + γ·M + δ·R. The weight coefficients α, β, γ, and δ are normalized to [0, 1] using the Z-score, i.e., [α, β, γ, δ] = [0.35, 0.30, 0.25, 0.10], reflecting the student's mental burden in real time.

[0135] The system strives to maintain training difficulty within the "zone of proximal development" or "fluid zone," an optimal range of challenges slightly above the student's current ability but still within manageable limits. This range is determined experimentally and typically corresponds to a cognitive load utilization rate of 70-85%. When cognitive load is detected to be excessive (>90%), the system appropriately reduces task complexity or provides additional support. When load is too low (<60%), the system increases the challenge or introduces secondary tasks to increase difficulty.

[0136] Based on error pattern and root cause analysis, the system generates targeted improvement suggestions, including targeted training content, practice methods and key focus areas.

[0137] For example, after detecting five consecutive landing attitude control ΔS < 0.05, the root cause analysis showed attention deficit (wattention = 0.72), which led to the generation of a basic strategy: multi-task hierarchical training, after optimization: difficulty = 0.65, environment = crosswind 0.6 + turbulence 0.4, dynamic rule: trigger voice feedback when pitch error > 3°

[0138] Trainees are required to: Practice landing in a simulated crosswind. When the pitch angle deviates too much, the joystick generates downward pressure. The system monitors cognitive load and reduces wind speed if it exceeds 0.85.

[0139] Monitor the trainee's training status: record skill improvement ΔS = 0.12, and use the TD algorithm to update the strategy network weights.

[0140] System integration and implementation architecture

[0141] The intelligent co-pilot system adopts a modular microservice architecture. Each functional module is designed as an independent microservice with a clearly defined API interface and responsibility boundary. This design enables each module to be independently developed, tested, and deployed, while achieving seamless collaboration through standardized interfaces. The core components of the system are deployed on a high-performance computing cluster, while the edge components are deployed locally on the training device, forming a hybrid edge-cloud architecture. Figure 6 As shown, balancing performance and latency requirements, deployment and scaling utilize containerization technology (Docker) and an orchestration platform (Kubernetes), enabling the system to automatically scale computing resources based on load. A continuous integration / continuous deployment (CI / CD) pipeline ensures that new features and fixes are quickly and securely deployed to the production environment. The system supports blue-green deployment and canary release strategies to minimize update risks.

[0142] The API gateway serves as the central hub for communication between modules, responsible for request routing, load balancing, authentication and authorization, and protocol conversion. The gateway implements a JWT (JSON Web Token)-based authentication mechanism and fine-grained access control to ensure system security. All inter-module communication utilizes a combination of asynchronous message queues and RESTful APIs. The message queues are used for high-frequency data streams and event notifications, while the RESTful APIs are used for query and control operations. The system also implements a service discovery mechanism, enabling modules to dynamically discover and connect to other services, enhancing system resiliency.

[0143] The decision engine, as a core component, is responsible for coordinating the work of various modules and maintaining system state. The engine uses the Actor model for concurrent processing, with each training session managed by a dedicated Actor to ensure state consistency and isolation. System state is stored using an event sourcing model, recording all state change events rather than just the current state. This enables the system to reconstruct its complete state at any point in time, facilitating debugging, replay, and auditing. To ensure high availability, key components employ an active-standby redundancy design, with service continuity guaranteed through a heartbeat mechanism and automatic failover.

[0144] The system implements a real-time processing pipeline, and this performance goal is achieved through multi-level optimization: first, the key computation paths are implemented in C++ and optimized at a low level; second, the deep learning model uses TensorRT for inference acceleration and quantization optimization; finally, the system uses an in-memory data grid (Hazelcast) to cache frequently accessed data, reducing computation and query overhead.

[0145] The data persistence layer adopts a hybrid storage strategy. Time-series flight data is stored in a dedicated time-series database (InfluxDB), structured training records and student information are stored in a relational database (PostgreSQL), and semi-structured logs and events are stored in a document database (MongoDB). The observability of the system is achieved through a comprehensive monitoring and logging mechanism. Each microservice exposes health check endpoints and performance indicators, which are collected and analyzed by the central monitoring system (Prometheus). Distributed tracing (Jaeger) enables the team to track cross-service requests and identify performance bottlenecks. Centralized log management (ELK Stack) aggregates all service logs and supports complex log queries and analysis. The system also implements an automatic alarm mechanism to proactively notify the operation and maintenance team when abnormal patterns or performance degradation are detected.

[0146] In this embodiment, the decision-making closed loop based on state perception is: "During the training execution phase, the system uses a temporal convolutional network (TCN) to identify the flight phase (such as final approach) in real time, and outputs a risk score in combination with the risk prediction model; the decision engine integrates flight parameters, student characteristics, and situational characteristics into a 300-dimensional state vector, and generates auxiliary decision instructions through a deep reinforcement learning strategy network. It is characterized by:

[0147] It uses a hierarchical decision-making mechanism, a dual network architecture (policy network + target network) and Bayesian decision theory to achieve dynamic decision-making. The high-level network updates the strategy every 1-2 seconds, and the low-level network generates specific instructions at 50Hz.

[0148] When the Bayesian network confidence is less than 0.7, the conventional decision-making process is skipped and the conservative strategy is directly enabled.

[0149] The core of the engine is the reward function, which dynamically weighs three key factors: learning effect, safety factor, and degree of intervention. The weights are adjusted according to the training stage and student characteristics, that is, meta-learning adjustment is used to ensure that the reward function adapts to the training process; a structured decision-making benchmark is provided by constructing a three-dimensional intensity matrix; and the closed-loop update of the system enables the two to evolve together, and the system achieves continuous self-optimization.

[0150] Situational understanding module, the situational understanding module accurately understands the semantic meaning and potential risks of the current flight status. It builds a knowledge graph and expert system and maintains a structured knowledge base containing a large number of aviation entities (flight procedures, systems, restrictions, etc.) and their complex semantic relationships. It supports rule-based rapid reasoning and fuzzy logic processing, and uses deep time series models to accurately identify fine flight phases (12 phases) and operating modes. At the same time, the anomaly detection function can identify parameter combinations that deviate from normal modes. Combining physical models and statistical learning (LSTM), it predicts changes in flight parameters in the next 5-20 seconds. By calculating the distance and approach speed between the predicted trajectory and the multi-dimensional safety envelope, a standardized risk score (0-100) is generated in real time to provide early warning of potential dangers.

[0151] The voice interaction interface is implemented, which achieves a highly realistic co-pilot interaction experience through deep integration of natural language processing, emotional state recognition and crew collaboration logic.

[0152] Aviation-grade voice interaction accurately understands standard aviation terminology and non-standard expressions. Based on the Transformer model and fine-tuned in the aviation field, it ensures accurate recognition of command intent. Natural and clear speech synthesis provides context-appropriate feedback.

[0153] The multimodal emotion and stress perception system monitors the student's status in real time, accurately identifying the student's stress level (normal, mild, moderate, and high stress) by integrating and analyzing voice features (pitch, speaking speed, etc.) and physiological indicators (eye movements and optional heart rate). It captures stress trends and adapts to individual differences through personalized calibration, providing a key basis for adjusting training difficulty and supporting strategies.

[0154] Intelligent co-pilot role-playing: The system dynamically simulates co-pilot role behaviors based on standard operating procedures (SOPs), flight phases, and scenarios. It implements five collaboration modes (full assistance, guidance, supervision, backup, and minimal intervention) and intelligently switches modes based on student capabilities, task complexity, and training objectives to provide appropriate support.

[0155] Adaptive implementation of personalized training, personalized training can optimize the training path according to the characteristics of the students.

[0156] The system builds a learning curve model for each key skill, estimating parameter values ​​from historical training data using nonlinear regression. This personalized learning curve accurately reflects the learning characteristics of individual learners. Stagnation detection calculates recent performance, analyzes it, and generates adjustment recommendations. If performance falls below a threshold, strategy adjustments are triggered.

[0157] Error pattern identification and root cause analysis: Build a detailed error classification system and use a hierarchical clustering algorithm to identify common patterns in student errors. Discover seemingly different but essentially similar error patterns.

[0158] Dynamic cognitive load control helps the system understand the student's mental burden and optimize training difficulty. By constructing a comprehensive cognitive load assessment using multiple indicators, a weighted combination forms a unified cognitive load index that reflects the student's mental burden in real time. This approach aims to maintain training difficulty within the "zone of proximal development" or "fluid zone," defined as the optimal challenge range, slightly above the student's current ability but still within their control.

[0159] Based on error pattern and root cause analysis, the system generates targeted improvement suggestions, including targeted training content, practice methods and key focus areas.

[0160] Regarding system integration and implementation architecture, the intelligent co-pilot system adopts a modular microservices architecture, enabling independent development, testing, and deployment of each module. Core system components are deployed on a high-performance computing cluster, while edge components are deployed locally on training devices, forming a hybrid edge-cloud architecture that balances performance and latency requirements. Deployment and scalability utilize containerization technology (Docker) and an orchestration platform (Kubernetes), enabling the system to automatically scale computing resources based on load. A continuous integration / continuous deployment (CI / CD) pipeline ensures that new features and fixes are quickly and securely deployed to production environments. The system supports blue-green deployment and canary release strategies to minimize update risks.

[0161] The data persistence layer adopts a hybrid storage strategy. Time series flight data is stored in a dedicated time series database (InfluxDB), structured training records and student information are stored in a relational database (PostgreSQL), and semi-structured logs and events are stored in a document database (MongoDB).

[0162] The system implements comprehensive monitoring and log observability. Each microservice exposes health check endpoints and performance metrics, which are collected and analyzed by a central monitoring system (Prometheus). Distributed tracing (Jaeger) tracks cross-service requests and identifies performance bottlenecks. Centralized log management (ELK Stack) aggregates all service logs and supports complex log querying and analysis.

[0163] In another embodiment, the architecture in the decision engine can be replaced by a multi-head attention network / graph neural network; Thompson-based priority sampling can be replaced by Upper Confidence Bound (UCB); and soft-constrained hierarchical control can be replaced by a fuzzy logic controller.

[0164] In another embodiment, the speech feature (Δ / ΔΔMel spectrum) in the multimodal emotion perception technology can be replaced by wavelet transform features; the SVM stress classifier can be replaced by random forest / Bayesian network; the LSTM time series analysis can be replaced by the Transformer time series encoder

[0165] In another embodiment, in the personalized training module, the stagnation detection component of the power-law learning curve can be replaced with Bayesian change point detection, Monte Carlo path optimization can be replaced with Bayesian optimization, and dynamic parameter binding (environment / difficulty / trigger conditions) can be replaced with genetic algorithm parameter optimization. In the cognitive load differential control component, the control function can be replaced with a Sigmoid / PID controller, and the target interval can be replaced with dynamic interval adjustment based on reinforcement learning.

[0166] In the root cause-driven training strategy part, error clustering (Ward algorithm) can be replaced by spectral clustering / DBSCAN, root cause analysis can be replaced by causal discovery algorithm (PC-algorithm) instead of neural network, and reinforcement learning decision tree can be replaced by case-based reasoning (CBR).

[0167] Another embodiment of the present invention provides an assisting method for a flight simulation training assisting system based on an intelligent co-pilot, comprising:

[0168] The intelligent co-pilot system automatically adjusts its assistance level and method based on the main driver's behavior, performance and training stage, dynamically adjusting from full assistance to minimal intervention and adaptive learning;

[0169] Through speech recognition and natural language processing, natural communication and interaction with the primary pilot are achieved, simulating the communication mode in a real cabin environment and conducting multimodal voice interaction;

[0170] The system identifies the current flight phase, training objectives, and potential risks in real time, provides corresponding professional support and suggestions, and performs emotional perception;

[0171] Based on historical training data, learning curves and error pattern analysis, the training content and difficulty are dynamically customized for each trainee to provide personalized training;

[0172] Provide progressive emergency assistance. In an emergency, the system provides graded assistance based on the trainees' abilities, from prompts and suggestions to direct takeover when necessary, to cultivate the trainees' ability to handle problems independently.

[0173] From the above description, it can be seen that the above embodiments of the present invention achieve the following technical effects:

[0174] The Intelligent Co-pilot Flight Simulation Training Assistance System is an integrated training platform designed to provide intelligent, personalized support for pilot training. Based on artificial intelligence technology, the system uses real-time data analysis and adaptive learning algorithms to simulate the functions and behaviors of a real co-pilot, providing targeted guidance to student pilots. The system consists of six tightly integrated functional modules, centered around a decision-making engine, forming a complete training ecosystem. This modular design ensures both system flexibility and scalability, while also enabling independent optimization and upgrades of each functional component.

[0175] Intelligent decision-making is deeply enhanced. Existing systems mostly use static rules or shallow machine learning, which makes it difficult to dynamically balance safety and teaching needs. This patent achieves dual-time-scale precise control through a layered reinforcement learning architecture (high-level policy network + low-level execution network). Combined with Thompson Bayesian exploration of adaptive student characteristics, it ensures real-time performance (50Hz response) while ensuring that the intervention strategy is both targeted and robust.

[0176] Adaptive training intensity selection. Traditional systems rely on instructor experience to pre-set training paths and are unable to dynamically adapt to student status. This patented system pioneers a meta-learning-driven reward function and a three-dimensional intervention intensity matrix (student level × flight phase × mission complexity), enabling autonomous evolution of training strategies from "safety guarantee" to "performance breakthrough." Combined with differential cognitive load control and root-cause-driven stagnation response, training intensity is consistently precisely matched to the student's "zone of proximal development," accelerating skill acquisition compared to fixed-difficulty approaches.

[0177] Multi-module collaborative optimization: Existing systems operate in isolation, leading to cognitive overload and training disconnection. This patented technology uses prioritized experience replay (TD-error weighted sampling) to close the data loop between the decision engine and personalized training modules, prioritizing learning of safety events (such as ground proximity warnings). Simultaneously, the semantic context understanding engine (knowledge graph + temporal convolutional network) analyzes operational intent in real time, reducing trainees' cognitive load and forming an intelligent enhancement cycle of "perception-decision-execution-optimization."

[0178] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A flight simulation training auxiliary system based on intelligent co-pilot, characterized in that: include: Data acquisition module, used for flight data collection and academy operation monitoring; Voice interaction interface, used for multimodal state analysis voice interaction; Learner model module for skill level assessment and learning style identification; Emotion perception module, used for flight experience recognition and risk prediction analysis; Personalized training module, used for training process monitoring, training result analysis and personalized training adjustment; Decision engine module, used for deep reinforcement learning models and assisting policy generators.

2. The flight simulation training assistance system based on intelligent co-pilot according to claim 1, characterized in that: The data acquisition module is responsible for obtaining key data during the training process. Through multiple sensor networks and simulator API interfaces, it captures data in real time, including a set of flight parameters such as speed, altitude, attitude, and position. It also records the trainee's operational behavior, including the joystick and throttle input mode, the interactive operation of the instrument panel, and the execution order and timing of the flight program.

3. The flight simulation training assistance system based on intelligent co-pilot according to claim 1, characterized in that: The acquisition module uses a signal processing algorithm, a Kalman filter and a wavelet transform technology to perform real-time preprocessing and noise filtering on the raw data. The acquisition module also uses a data integrity check mechanism to ensure that the data is not damaged or lost during transmission through a cyclic redundancy check and a checksum algorithm.

4. The flight simulation training auxiliary system based on intelligent co-pilot according to claim 1, characterized in that: The student model module is responsible for building and maintaining the digital capability portrait of each student. The student model module is based on Bayesian statistics and hidden Markov models. By analyzing the historical training data of the students, a multi-dimensional capability assessment is formed. The student model module includes key dimensions such as technical operation capability, decision-making efficiency, emergency response, resource management and program execution, and assigns quantitative indicators and uncertainty intervals to each dimension.

5. The flight simulation training auxiliary system based on intelligent co-pilot according to claim 1, characterized in that: In terms of learning style identification, the student model module uses a recursive neural network to analyze students' response patterns and decision-making tendencies in different training scenarios, including micro-behavioral characteristics such as risk preferences, hesitation patterns, and attention allocation habits.

6. The flight simulation training auxiliary system based on intelligent co-pilot according to claim 1, characterized in that: The student model adopts an online learning mechanism and can continuously update and optimize itself as new training data accumulates. The updating process adopts an incremental learning algorithm. The weight of new data is dynamically adjusted according to the time decay function to reflect the student's latest ability status. At the same time, a self-calibration mechanism is adopted to regularly verify the accuracy of the model through standardized test tasks and make necessary parameter adjustments.

7. The flight simulation training auxiliary system based on intelligent co-pilot according to claim 1, characterized in that: The situational understanding module provides the system with deep flight context perception capabilities, accurately understands the semantic meaning and potential risks of the current flight status, and lays the foundation for intelligent decision-making. The situational understanding module is implemented through knowledge graphs and expert systems, temporal convolutional network analysis and risk prediction models. By integrating domain knowledge, real-time status analysis and future risk prediction, it achieves a dynamic, comprehensive and semantic understanding of training scenarios, providing key situational input for subsequent intelligent intervention and teaching strategies.

8. The flight simulation training auxiliary system based on intelligent co-pilot according to claim 1, characterized in that: The personalized training module builds a data-driven personalized training closed loop by deeply integrating intelligent modeling of learning curves, in-depth diagnosis of root causes of errors, and precise regulation of cognitive load; The voice interaction interface achieves a highly realistic co-pilot interaction experience by deeply integrating natural language processing, emotional state recognition and crew collaboration logic.

9. The flight simulation training auxiliary system based on intelligent co-pilot according to claim 1, characterized in that: The decision engine module dynamically decides whether to intervene, the type and intensity of intervention based on the real-time status, balancing the students' learning effect and flight safety, while avoiding excessive intervention that damages the students' autonomous learning ability. The engine is responsible for integrating information from various modules of the simulator and generating intelligent auxiliary decisions. The real-time status includes flight parameters, student status and situational characteristics.

10. An assisting method for a flight simulation training assisting system based on an intelligent co-pilot, based on the flight simulation training assisting system based on an intelligent co-pilot according to any one of claims 1 to 9, characterized in that: include: The intelligent co-pilot system automatically adjusts its assistance level and method based on the main driver's behavior, performance and training stage, dynamically adjusting from full assistance to minimal intervention and adaptive learning; Through speech recognition and natural language processing, natural communication and interaction with the primary pilot are achieved, simulating the communication mode in a real cabin environment and conducting multimodal voice interaction; The system identifies the current flight phase, training objectives, and potential risks in real time, provides corresponding professional support and suggestions, and performs emotional perception; Based on historical training data, learning curves and error pattern analysis, the training content and difficulty are dynamically customized for each trainee to provide personalized training; Provide progressive emergency assistance. In an emergency, the system provides graded assistance based on the trainees' abilities, from prompts and suggestions to direct takeover when necessary, to cultivate the trainees' ability to handle problems independently.

Images