Flight training system

Abstract

The present invention relates to the technical field of flight training, and more particularly to a flight training system. A flight joystick receives operational action data input by a target user based on preset training instructions; a button control panel receives button action data input by the target user based on preset training instructions; an electronic device receives parameter setting data for flight parameters input by the target user based on preset training instructions; a multi-axis gravity seat performs six-degree-of-freedom motion based on the operational action data, button action data, and parameter setting data, thereby simulating the flight attitude of an aircraft; and the electronic device generates a target virtual driving scene based on the operational action data, button action data, and parameter setting data, and controls a dynamic scene display device to display the target virtual driving scene. This achieves the integration of the target virtual driving scene with the actual operating process, thereby improving the training effect during flight training.

Description

Technical Field

[0001] The present invention relates to the technical field of flight training, and in particular to a flight training system. Background Art

[0002] Flight training, as the core professional training for aviation forces, plays a key role in improving pilots' technical and tactical proficiency in operating aircraft and employing onboard equipment and weapons. This encompasses pilot training, as well as the training of specialized personnel such as navigators, communicators, and gunners, and includes coordination training among crew members.

[0003] Traditional flight training methods usually involve conducting flight training in training simulators based on fixed driving scenarios. This lacks the integration of driving scenarios with actual operations and cannot provide feedback on the user's training actions, resulting in poor training results.

[0004] Therefore, how to improve the training effect during flight training has become an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the present invention provides a flight training system to solve the problem of how to improve the training effect during the flight training process.

[0006] In a first aspect, the present invention provides a flight training system, comprising a multi-axis gravity seat, a flight control stick, a button control panel, a parameter setting interface, a dynamic scene display device, and an electronic device; wherein the multi-axis gravity seat, the flight control stick, and the dynamic scene display device are all communicatively connected to the electronic device; wherein:

[0007] The flight joystick is used to receive operation action data input by the target user based on preset training instructions. The operation action data includes the execution order corresponding to each operation action and the operation force corresponding to each operation action;

[0008] Button control panel, used to receive button action data input by the target user based on preset training instructions;

[0009] An electronic device, configured to receive parameter setting data for setting flight parameters input by a target user based on preset training instructions;

[0010] A multi-axis gravity seat, which is used to perform six-degree-of-freedom motion based on operation action data, button action data, and parameter setting data, thereby simulating the flight attitude of the aircraft;

[0011] The electronic device is also used to generate a target virtual driving scene based on the operation action data, button action data and parameter setting data, and control the dynamic scene display device to display the target virtual driving scene.

[0012] In the flight training system provided by the embodiments of the present application, the flight joystick receives operational action data input by the target user based on preset training instructions. The button control panel also receives button action data input by the target user based on preset training instructions. This allows the flight training system to accurately capture subtle differences in the target user's operations for flight attitude adjustment, power control, and other operations, providing accurate data support for subsequent simulations. The button action data received by the button control panel records the user's operations on various aircraft systems and function switches. The parameter setting data received by the electronic equipment includes settings for parameters such as flight altitude, speed, and heading. The comprehensive collection of this multi-dimensional data fully reproduces the user's operational process during flight training, enabling the system to accurately understand the user's operational intent and laying a solid foundation for simulating flight attitude and generating virtual scenes. The multi-axis gravity seat is used to perform six-degree-of-freedom motion based on the operational action data, button action data, and parameter setting data, thereby simulating the aircraft's flight attitude. This allows the target user to physically experience changes in flight attitude. This highly realistic physical simulation breaks the limitations of traditional two-dimensional simulation training, making the target user feel as if they were in a real cockpit. This greatly enhances the immersive experience, helps the target user quickly familiarize themselves with the physical sensations of flight, and improves their ability to judge and control flight attitude. Based on operational action data, button action data, and parameter setting data, the electronic device generates the target virtual driving scene and controls the dynamic scene display device to display the target virtual driving scene. This ensures the accuracy of the generated target virtual driving scene and integrates the target virtual driving scene with the actual operation process. The various components of the flight training system communicate with the electronic device to achieve data exchange and collaborative operation. This collaborative mechanism makes the training process more fluid and realistic, and every user's operation receives timely and accurate feedback, promoting the target user to quickly learn and master flight skills, thereby improving training effectiveness.

[0013] In an optional embodiment, the flight training system further includes an airflow control system, which includes a plurality of airflow valves and an airflow pressure sensor; the airflow control system is communicatively connected to the electronic device, wherein:

[0014] An electronic device, configured to determine, based on the preset training instructions, external environmental data corresponding to the preset training instructions; and to determine, based on the external environmental data and the flight attitude, a target airflow direction and a target airflow velocity corresponding to the aircraft;

[0015] The airflow control system is used to detect the current airflow direction and current airflow speed based on each airflow pressure sensor; and control the opening or closing of each airflow valve according to the difference between the target airflow direction and target airflow speed and the current airflow direction and current airflow speed.

[0016] In an optional embodiment, the multi-axis gravity seat includes a superconducting magnetic levitation device, a six-degree-of-freedom electric seat, and an adjustable seat belt array; wherein:

[0017] The six-degree-of-freedom electric seat is installed on a superconducting magnetic levitation device. The six-degree-of-freedom electric seat is suspended above the superconducting magnetic levitation device through the Meissner effect generated by the superconducting material under a preset temperature environment;

[0018] The six-degree-of-freedom electric seat is equipped with a linear drive unit consisting of multiple sets of permanent magnets and electromagnetic coils to achieve six-degree-of-freedom movement;

[0019] The adjustable seatbelt array is connected to the six-degree-of-freedom electric seat through multiple independent electric tightening devices, adjusting the pressure applied to the target user according to the flight attitude.

[0020] In an optional embodiment, the flight training system further includes a data acquisition device, which is communicatively connected to the electronic device, wherein:

[0021] Data acquisition equipment, used to collect the target user's corresponding physiological time series data, EEG time series data, eye movement image data and facial image data during the training process;

[0022] An electronic device, configured to input physiological time series data, EEG time series data, eye movement image data, and facial image data into a preset workload recognition model, and output a current workload level corresponding to a target user;

[0023] The electronic device is also used to generate an alternative virtual driving scenario based on preset training instructions, operation action data, button action data and parameter setting data; adjust the complexity of the alternative virtual driving scenario according to the target cognitive load level to generate a target virtual driving scenario.

[0024] In an optional embodiment, the preset workload identification model includes a first feature identification network and a second feature identification network, and an electronic device is configured to:

[0025] Inputting physiological time series data and EEG time series data into a first feature extraction network to generate time series modal features;

[0026] Inputting the facial image data and the eye movement image data into a second feature extraction network to generate visual modality features;

[0027] Fuse the temporal modal features and visual modal features to generate target fusion features;

[0028] Based on the target fusion features, the current workload level corresponding to the target user is output.

[0029] In an optional embodiment, the electronic device is configured to use the visual modality feature as a first query matrix and the temporal modality feature as a first key matrix and a first value matrix;

[0030] Calculating the first dependency weight of the visual modality feature on the temporal modality feature;

[0031] The temporal modality features are used as the second query matrix, and the visual modality features are used as the second key matrix and the second value matrix;

[0032] Calculating the second dependency weight of the temporal modality feature on the visual modality feature;

[0033] Multiply the time series modal feature by the first dependency weight to obtain the target time series feature;

[0034] Multiply the second dependency weight by the visual modality feature to obtain the target visual feature;

[0035] The target temporal features and target visual features are combined to generate target fusion features.

[0036] In an optional embodiment, the electronic device is configured to generate an initial virtual driving scenario based on preset training instructions;

[0037] Based on the initial virtual driving scene, an alternative virtual driving field is generated based on the operation action data, button action data and parameter setting data.

[0038] In an optional embodiment, the electronic device is configured to perform semantic recognition on a preset training instruction and extract key information from the preset training instruction; the key information includes at least one of a flight mission type, a flight area, and weather conditions;

[0039] Each key information is used as the initial node, the dependency relationship between key information is used as the initial edge, and attribute information is set for each initial node to construct the initial graph structure;

[0040] Perform convolution on the initial graph structure to obtain the initial node features corresponding to each initial node in the graph structure;

[0041] Based on the features of each initial node, an initial virtual driving scene is generated.

[0042] The in-depth training environment helps pilots better understand and master the key points of operating different flight missions under various conditions.

[0043] In an optional embodiment, the electronic device is configured to abstract the operation action data, the button action data, and the parameter setting data into update nodes, and establish update edge connections with each initial node in the initial graph structure;

[0044] According to the association between the operation action data, button action data, parameter setting data and preset training instructions, the weight and direction of each updated edge are determined, and the attribute information of each updated node is updated to generate the target graph structure;

[0045] Perform convolution on the target graph structure to obtain the target node features corresponding to each target node in the graph structure; the target nodes include each initial node and each update node;

[0046] The feature matrix of each target node is input into the preset recurrent neural network at each time step to obtain the current target state feature;

[0047] Based on the current target state characteristics, the initial virtual driving scene is adjusted to generate a backup virtual driving scene.

[0048] In an optional embodiment, the electronic device is further configured to input the physiological time series data, the EEG time series data, the eye movement image data, the facial image data, and the target cognitive load level into the meta-learner;

[0049] The meta-learner obtains an initial cognitive load-scenario response model, which consists of an input layer, a hidden layer, and an output layer. The input layer receives data, and its number of neurons is set based on the data's feature dimensions. The hidden layer, composed of multiple layers of neurons, processes the data through complex connection weights. The output layer outputs the final result.

[0050] The meta-learner adjusts the initial cognitive load-scenario response model based on physiological time series data, EEG time series data, eye movement image data, and facial image data to generate a target cognitive load-scenario response model corresponding to the target user;

[0051] The target cognitive load level is input into the target cognitive load-scenario response model, and the scenario complexity adjustment parameters are output. The scenario complexity adjustment parameters include fault type adjustment, fault frequency adjustment, and environment complexity adjustment:

[0052] The complexity of the backup virtual driving scene is adjusted based on the scene complexity adjustment parameter to generate a target virtual driving scene. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1is a schematic structural diagram of a flight training system according to an embodiment of the present invention;

[0055] Figure 2 is a schematic structural diagram of another flight training system according to an embodiment of the present invention;

[0056] Figure 3 is a schematic structural diagram of another flight training system according to an embodiment of the present invention;

[0057] Figure 4 is a schematic diagram of a process of outputting a current workload level corresponding to a target user according to an embodiment of the present invention;

[0058] Figure 5 is a schematic diagram of a process for generating a backup virtual driving scene according to an embodiment of the present invention;

[0059] Figure 6 4 is a flow chart of generating a target virtual driving scene according to an embodiment of the present invention.

[0060] Among them: a multi-axis gravity seat 1; a superconducting magnetic levitation device 11; a six-degree-of-freedom electric seat 12; an adjustable seat belt array 13; a flight joystick 2; a button control panel 3; a parameter setting interface 4; a dynamic scene display device 5; an electronic device 6; an airflow control system 7; an airflow valve 71; and an airflow pressure sensor 72. DETAILED DESCRIPTION

[0061] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0062] The present application embodiment provides a flight training system, such as Figure 1 As shown, the flight training system includes a multi-axis gravity seat 1, a flight control stick 2, a button control panel 3, a parameter setting interface 4, a dynamic scene display device 5, and an electronic device 6; wherein the multi-axis gravity seat 1, the flight control stick 2, and the dynamic scene display device 5 are all communicatively connected to the electronic device 6; wherein:

[0063] The flight joystick 2 is used to receive operation action data input by the target user based on preset training instructions. The operation action data includes the execution order corresponding to each operation action and the operation force corresponding to each operation action.

[0064] The button control panel 3 is used to receive button action data input by the target user based on preset training instructions.

[0065] The electronic device 6 is used to receive parameter setting data for setting flight parameters input by the target user based on the preset training instructions.

[0066] The multi-axis gravity seat 1 is used to perform six-degree-of-freedom motion based on operation action data, button action data and parameter setting data, thereby simulating the flight attitude of an aircraft.

[0067] The electronic device 6 is also used to generate a target virtual driving scene based on preset training instructions, operation action data, button action data and parameter setting data, and control the dynamic scene display device 5 to display the target virtual driving scene.

[0068] Specifically, the flight training system consists of a multi-axis gravity seat 1, a flight control stick 2, a button control panel 3, a parameter setting interface 4, a dynamic scene display device 5, and an electronic device 6. The multi-axis gravity seat 1, flight control stick 2, and dynamic scene display device 5 establish a communication connection with the electronic device 6 to ensure real-time and accurate data transmission. This allows the electronic device 6 to obtain user operation information and control related equipment to respond, thus building an organic and coordinated training system.

[0069] The flight joystick 2 serves as the core input device for the user to control the aircraft. It has multiple built-in sensors (such as force sensors and angle sensors). When the target user performs operations according to preset training instructions, the force sensor captures the corresponding operating force in real time, and the angle sensor records the execution sequence and direction changes of the operations. These physical operations are converted into electrical signals and encoded into operation action data. For example, when a user pulls the joystick to take off, the joystick converts information such as the force applied, the angle change, and the time sequence of the operations into operation action data. This information is transmitted to the electronic device 6 via a communication connection, providing basic data for subsequent scene generation and seat simulation.

[0070] The button control panel 3 is equipped with various function buttons corresponding to the control of different aircraft systems and functions. When a user presses a button according to preset training instructions, the circuit system of the button control panel 3 detects the change in the button's pressed state and converts it into button action data, including information such as the button's identity and the time the button was pressed. For example, if a user presses the button to activate the autopilot mode, the button control panel 3 encodes the button action information and transmits it to the electronic device 6 via a communication link. The electronic device 6 then understands the user's intended operation of the aircraft system function.

[0071] Electronic device 6 receives parameter setting data for flight parameters entered by the target user via parameter setting interface 4. Parameter setting interface 4 is typically presented in the form of a touch screen, knobs, or a combination of buttons. The user sets parameters such as flight altitude, speed, heading, and engine power on this interface. The input data is converted into digital signals by the interface's signal processing module and transmitted to electronic device 6. Electronic device 6 analyzes and stores this data, which serves as an important basis for generating virtual driving scenarios and simulated flight attitudes.

[0072] After receiving the preset training instructions, operation action data, button action data, and parameter setting data, the electronic device 6 first performs data preprocessing. Then, the electronic device 6 generates a target virtual driving scene based on the preprocessed preset training instructions, operation action data, button action data, and parameter setting data.

[0073] After generating the target virtual driving scene, electronic device 6 transmits the target scene data to dynamic scene display device 5 (e.g., a large display screen or virtual reality headset) via a graphics interface (e.g., HDMI or DisplayPort). After receiving the data, dynamic scene display device 5 decodes and renders the data using its own display driver and graphics processing unit (GPU), presenting the target virtual driving scene to the user in the form of an image or video. By viewing the display device, the user experiences a realistic flight environment, which, combined with the physical simulation of the multi-axis gravity chair 1, provides a highly immersive flight training experience.

[0074] Through the above principles and workflows, the flight training system realizes a complete closed loop from user operation input to virtual scene presentation and physical simulation, providing pilots with an efficient and realistic training environment, helping pilots improve their flying skills and ability to cope with various flight situations.

[0075] In the flight training system provided by the present embodiment, the flight joystick 2 receives operational action data input by the target user based on preset training instructions. The button control panel 3 also receives button action data input by the target user based on preset training instructions. This allows the flight training system to capture subtle differences in the target user's operations for flight attitude adjustment, power control, and other operations, providing accurate data support for subsequent simulations. The button action data received by the button control panel 3 records the user's operations on various aircraft systems and function switches. The electronic device 6 receives parameter setting data, including flight altitude, speed, and heading settings. This comprehensive collection of multi-dimensional data fully reproduces the user's operational process during flight training, enabling the system to accurately understand the user's operational intent and laying a solid foundation for flight attitude simulation and virtual scene generation. The multi-axis gravity seat 1 is used to perform six-degree-of-freedom motion based on the operational action data, button action data, and parameter setting data, thereby simulating the aircraft's flight attitude. This allows the target user to physically experience changes in flight attitude. This highly realistic physical simulation breaks the limitations of traditional two-dimensional simulation training, making the target user feel as if they are in a real cockpit, greatly enhancing the immersiveness of the training, helping the target user to quickly familiarize themselves with the physical sensations during flight and improve their ability to judge and control flight attitude. Based on the operation action data, button action data, and parameter setting data, the electronic device 6 generates a target virtual driving scene and controls the dynamic scene display device 5 to display the target virtual driving scene. This ensures the accuracy of the generated target virtual driving scene and integrates the target virtual driving scene with the actual operation process. The various components of the flight training system communicate with the electronic device 6 to achieve data exchange and collaborative operation. This collaborative mechanism makes the training process more fluid and realistic, and every operation of the target user receives timely and accurate feedback, which promotes the target user to quickly learn and master flight skills, thereby improving the training effect during the flight training process.

[0076] In an optional embodiment of the present application, Figure 2 As shown, the flight training system further includes an airflow control system 7, which includes a plurality of airflow valves 71 and an airflow pressure sensor 72; the airflow control system 7 is in communication with the electronic device 6, wherein:

[0077] The electronic device 6 is used to determine the external environment data corresponding to the preset training instruction based on the preset training instruction; and determine the target airflow direction and target airflow speed corresponding to the aircraft based on the external environment data and the flight attitude.

[0078] The airflow control system 7 is used to detect the current airflow direction and the current airflow speed based on each airflow pressure sensor 72; and control the opening or closing of each airflow valve 71 according to the difference between the target airflow direction and the target airflow speed and the current airflow direction and the current airflow speed.

[0079] Specifically, during actual flight, the aircraft's external environment (such as weather and altitude) and its flight attitude (such as pitch, roll, and yaw) affect the airflow around the aircraft. Therefore, multiple airflow valves 71 are installed on the sides of the multi-axis gravity seat 1 to simulate the direct effects of airflow on the human body during flight (such as wind pressure during climbs and dives). Airflow pressure sensors 72 can be installed around the multi-axis gravity seat 1 to monitor airflow changes around the seat in real time and provide feedback to the electronic equipment to adjust simulation parameters. This directly affects the trainee's body, enhancing the realism of gravity and acceleration changes (such as simulating airflow impact and crosswind interference). By simulating external environmental data corresponding to preset training instructions and combining it with the aircraft's flight attitude to determine the target airflow direction and speed, the trainee can experience airflow changes that are close to those experienced during real-life flight, enhancing the realism and immersion of the training. Furthermore, combined with the multi-axis gravity seat 1's six degrees of freedom, this provides multimodal feedback of "motion + airflow," enhancing the fidelity of the training.

[0080] Airflow control system 7 uses airflow pressure sensor 72 to detect the current airflow direction and speed in real time, compares them with target values, and adjusts the state of airflow valve 71 based on the difference, forming a feedback control loop. This feedback mechanism continuously optimizes the airflow state, keeping it as close to the target value as possible, ensuring the accuracy and stability of the simulation.

[0081] Specific workflow: Electronic device 6 first receives a preset training instruction and determines the corresponding external environmental data based on the instruction. For example, if the training instruction simulates a flight scenario on a sunny day at an altitude of 5,000 meters, electronic device 6 will obtain the corresponding environmental parameters such as weather, air pressure, and temperature.

[0082] Then, electronic device 6 combines the determined external environmental data with the aircraft's simulated flight attitude as measured by the multi-axis gravity chair 1 to calculate the target airflow direction and velocity corresponding to the aircraft's current state using a specific algorithm or model. For example, if the aircraft is in a climbing attitude and there is a certain crosswind, the airflow direction and velocity under these combined influences must be calculated.

[0083] Airflow pressure sensors 72 in the airflow control system 7 continuously detect the current direction and speed of the airflow. These sensors are distributed in relevant locations throughout the training system and can accurately sense real-time changes in the airflow and transmit the detected data to the electronic device 6.

[0084] Electronic equipment 6 compares the target airflow direction and speed with the current airflow direction and speed, calculating the difference between the two. Airflow control system 7 controls the opening and closing of individual airflow valves 71 based on these differences. If the current airflow speed is lower than the target speed, more airflow valves 71 may be opened to increase the airflow rate. If the airflow direction deviates from the target, the airflow direction is altered by adjusting the positions of airflow valves 71, gradually approaching the target value. Through continuous monitoring, comparison, and adjustment, the airflow state is continuously optimized, providing the trainee with a realistic airflow experience that matches the current flight conditions.

[0085] In the flight training system provided by the present embodiment, electronic equipment 6 determines the external environmental data corresponding to the preset training instructions based on the preset training instructions. Furthermore, based on the external environmental data and the flight attitude, the corresponding target airflow direction and target airflow velocity are determined. This allows precise setting of airflow conditions based on different training scenarios and requirements. Airflow control system 7 detects the current airflow direction and velocity using airflow pressure sensors 72 and controls the opening or closing of airflow valves 71 based on the difference between the target airflow direction and velocity and the current airflow direction and velocity. This system accurately simulates the airflow conditions encountered by an aircraft in various flight environments and flight attitudes, such as those during takeoff, landing, and encountering turbulence, allowing pilots to experience realistic airflow effects and enhancing the immersiveness and authenticity of training. Furthermore, when simulating dangerous airflow conditions, pilots can familiarize themselves with response strategies in a safe training environment, avoiding the dangers of encountering similar situations in actual flight due to lack of experience. Furthermore, the precise control of airflow control system 7 prevents damage to training equipment or personnel due to improper airflow simulation. During training, pilots can truly feel the impact of airflow on flight, better understand and master the aircraft's control characteristics under different airflow conditions, which helps improve flying skills and reaction speed, makes the training effect closer to actual flight, and shortens the transition time from training to actual flight.

[0086] In an optional embodiment of the present application, Figure 3 As shown, the multi-axis gravity chair 1 includes a superconducting magnetic levitation device 11, a six-degree-of-freedom electric seat 12, and an adjustable safety belt array 13; wherein:

[0087] The six-degree-of-freedom electric seat 12 is mounted on the superconducting magnetic levitation device 11. The six-degree-of-freedom electric seat 12 is suspended above the superconducting magnetic levitation device 11 by the Meissner effect generated by the superconducting material under a preset temperature environment.

[0088] The six-degree-of-freedom electric seat 12 is equipped with a linear drive unit consisting of multiple sets of permanent magnets and electromagnetic coils to achieve six-degree-of-freedom movement;

[0089] The adjustable seat belt array 13 is connected to the six-degree-of-freedom electric seat 12 through multiple independent electric tightening devices, and the pressure applied to the target user is adjusted according to the flight posture.

[0090] Specifically, the superconducting magnetic levitation device 11 achieves magnetic levitation by utilizing the Meissner effect generated by superconducting materials under a preset temperature environment. The Meissner effect refers to the phenomenon in which superconducting materials, when in their superconducting state, completely repel magnetic fields, preventing magnetic field lines from penetrating the superconductor. This creates an upward levitation force, allowing the six-degree-of-freedom electric seat 12 to stably levitate above the superconducting magnetic levitation device 11. This levitation method reduces mechanical friction and improves movement precision and flexibility. The six-degree-of-freedom electric seat 12 achieves six degrees of freedom by employing multiple linear drive units composed of permanent magnets and electromagnetic coils. According to the Ampere force principle, when current flows through an electromagnetic coil, it is acted upon by a force in a magnetic field, and the permanent magnets provide a stable magnetic field. By precisely controlling the magnitude and direction of the current in the electromagnetic coil, the linear drive unit can generate forces of varying directions and magnitudes, thereby driving the six-degree-of-freedom electric seat 12 to translate (move along the X, Y, and Z axes) and rotate (rotate about the X, Y, and Z axes) in space, simulating various aircraft flight attitudes.

[0091] The adjustable seat belt array 13 is connected to the six-degree-of-freedom electric seat 12 through multiple independent electric tightening devices. The principle is that based on the flight posture simulated by the six-degree-of-freedom electric seat 12, the electronic device 6 calculates the restraint force required by the target user in different postures, and then controls the electric tightening device to adjust the tightness of the seat belt, thereby applying appropriate pressure to the target user, allowing the user to feel the changes in gravity and acceleration that match the flight posture, thereby enhancing the realism and immersion of flight training.

[0092] The specific workflow is as follows: When the flight training system is activated, the superconducting magnetic levitation device 11 cools the superconducting material to a preset temperature, causing it to enter a superconducting state and generate the Meissner effect. At this point, the six-degree-of-freedom electric seat 12, under the levitation force generated by the Meissner effect, stably levitates above the superconducting magnetic levitation device 11, providing a frictionless support foundation for subsequent movement.

[0093] Electronic device 6 calculates the desired simulated aircraft flight attitude based on pre-set flight training instructions and related data (such as operational action data, button action data, and parameter setting data). Electronic device 6 then sends control signals to the linear drive unit of the six-degree-of-freedom power seat 12. By controlling the current in the electromagnetic coil, the permanent magnet interacts with the electromagnetic coil, generating corresponding forces and torques. This drives the six-degree-of-freedom power seat 12 to move in six degrees of freedom, accurately simulating various aircraft flight attitudes, such as takeoff, landing, turns, climbs, and dives.

[0094] Furthermore, while the six-degree-of-freedom powered seat 12 simulates flight postures, the electronic device 6 monitors the seat's posture in real time and, based on a pre-set algorithm, calculates the pressure required by the target user in their current posture. The electronic device 6 then sends commands to the electric tensioning devices of the adjustable seatbelt array 13, controlling them to tighten or loosen the belts. Each electric tensioning device operates independently, precisely adjusting the belt tension based on the force requirements of different body parts. This ensures the appropriate pressure is applied to the target user in different flight postures, providing a realistic flight experience.

[0095] The flight training system provided in the present application features a six-degree-of-freedom (6DOF) powered seat 12 mounted on a superconducting magnetic levitation device 11. The Meissner effect, generated by superconducting materials under a preset temperature environment, allows the six-degree-of-freedom powered seat 12 to levitate above the superconducting magnetic levitation device 11, achieving contactless suspension support. This suspension method avoids the friction and wear associated with traditional mechanical supports, reduces energy loss, improves system stability and reliability, and lowers maintenance costs. The six-degree-of-freedom powered seat 12 is equipped with a linear drive unit consisting of multiple sets of permanent magnets and electromagnetic coils to achieve six degrees of freedom. This allows for precise control of the seat's movement in all six degrees of freedom, including forward, backward, left, right, and vertical translation, as well as pitch, roll, and yaw rotations. This enables the seat to accurately simulate various flight attitudes, providing users with a highly realistic flight experience and helping pilots better adapt to the changes in attitude experienced during actual flight during training. The linear drive unit offers rapid response, rapidly adjusting the seat's position and attitude based on changes in flight attitude, tracking the dynamics of simulated flight in real time, reducing latency and enhancing user immersion and training effectiveness. In addition, it can withstand large loads to meet the needs of users of different body shapes. At the same time, when simulating high-intensity flight movements, it can also ensure the stability and safety of the seat, ensuring the safety of users during training. The adjustable seat belt array 13 is connected to the six-degree-of-freedom electric seat 12 through multiple independent electric tightening devices, and the pressure applied to the target user is adjusted according to the flight posture. This can provide users with personalized constraints and support, ensuring that the user's body can be properly fixed in different flight postures, improving comfort and safety. Dynamically adjusting the pressure of the seat belt according to the flight posture can allow users to more realistically feel the changes in gravity and the stress on the body during flight, further enhancing the immersion of simulated flight and allowing users to better adapt to the physiological feelings of actual flight during training.

[0096] During the simulated flight, especially when performing some difficult or dangerous flying maneuvers, the adjustable safety belt array 13 can restrain the user in a timely and effective manner, preventing the user from being displaced or injured in the seat, and providing additional safety protection for the user.

[0097] In an optional embodiment of the present application, the flight training system further includes a data acquisition device, which is communicatively connected to the electronic device 6, wherein:

[0098] Data acquisition equipment, used to collect the target user's corresponding physiological time series data, EEG time series data, eye movement image data and facial image data during the training process;

[0099] The electronic device 6 is used to input the physiological time series data, the EEG time series data, the eye movement image data and the facial image data into a preset workload recognition model, and output the current workload level corresponding to the target user.

[0100] The electronic device 6 is further used to generate an alternative virtual driving scenario based on preset training instructions, operation action data, button action data, and parameter setting data; and to adjust the complexity of the alternative virtual driving scenario according to the target cognitive load level to generate a target virtual driving scenario.

[0101] Specifically, the data acquisition system may include a wearable human physiological recorder, a water electrode EEG system, a wearable eye tracker, and a camera. The wearable human physiological recorder may be an ErgoLAB smart wearable human physiological recorder, a wearable multi-parameter vital sign comprehensive detector that can be worn anywhere on the human body. It can monitor physiological indicators such as RESP respiratory rate, HR heart rate, ECG electrocardiogram (ECG) changes, EDA electrical skin conduction (EDA) changes, PPG pulse changes, and EMG electromyography in real time. It can also extract human posture changes and GPS spatiotemporal behavior trajectory and spatial location data in real time. Thus, electronic device 6 can collect physiological time series data corresponding to the target user based on the ErgoLAB smart wearable human physiological recorder.

[0102] The water electrode EEG system can be a semi-dry EEG water electrode EEG system. The semi-dry EEG water electrode EEG system consists of 32 EEG electrodes, a reference electrode, a wireless EEG signal amplifier, an EEG cap, and electrode pads. It has a sampling rate of up to 32 kHz, a resolution of 24 bits, and supports programmable signal amplification of 10-1000 times. Thus, electronic device 6 can collect EEG time series data based on the semi-dry EEG water electrode EEG system.

[0103] The wearable eye tracker can be a TobiiGlasses2 wearable eye tracker, which features wireless real-time observation and is designed for research in real-world environments. The eye tracker has a sampling rate of 50 Hz or 100 Hz. Thus, electronic device 6 can collect eye movement image data corresponding to the target user using the TobiiGlasses2 wearable eye tracker.

[0104] The camera device can collect facial image data corresponding to the target user.

[0105] Then, the electronic device 6 inputs the physiological time series data, the EEG time series data, the eye movement image data and the facial image data into a preset workload recognition model, and outputs the current workload level corresponding to the target user.

[0106] In addition, the electronic device 6 generates a backup virtual driving scene based on preset training instructions, operation action data, button action data and parameter setting data; adjusts the complexity of the backup virtual driving scene according to the target cognitive load level to generate a target virtual driving scene.

[0107] In the flight training system provided in the embodiments of the present application, a data acquisition device collects physiological time-series data, EEG time-series data, eye movement image data, and facial image data corresponding to a target user during training. An electronic device 6 inputs the physiological time-series data, EEG time-series data, eye movement image data, and facial image data into a preset workload identification model and outputs the target user's corresponding current workload level. The accuracy of the output current workload level is guaranteed. Accurate workload identification enables the training system to understand the user's stress and cognitive state during training in real time, preventing excessive training difficulty from causing excessive anxiety and fatigue, which could affect training effectiveness, and preventing training from being too low, which could make the training lack challenging. The system is also used to generate alternative virtual driving scenarios based on preset training instructions, operation action data, button action data, and parameter setting data, ensuring the accuracy of the generated alternative virtual driving scenarios. Furthermore, the system adjusts the complexity of the alternative virtual driving scenarios based on the target cognitive load level to generate the target virtual driving scenario. When the user's workload is low, the system increases the scenario complexity, such as introducing complex weather conditions and multi-system failure combinations, to increase the difficulty of training and stimulate the user's learning potential. When the user's workload is too high, the system reduces the scenario complexity, reduces interference factors, simplifies the task process, helps users relieve stress, and maintain a good training state. This dynamic adjustment mechanism ensures that the training scenario is always matched to the user's cognitive load, enhancing user immersion and participation, effectively improving training results, and making training more targeted and practical.

[0108] In an optional embodiment of the present application, the preset workload recognition model includes a first feature recognition network and a second feature recognition network, such as Figure 4 As shown, the above-mentioned “inputting physiological time series data, EEG time series data, eye movement image data, and facial image data into a preset workload recognition model and outputting the current workload level corresponding to the target user” may include the following steps:

[0109] Step S101: input physiological time series data and EEG time series data into a first feature extraction network to generate time series modal features.

[0110] Specifically, the first feature extraction network includes a first sub-feature extraction branch, a second sub-feature extraction branch, and a first sub-feature fusion network. The above step S101 may include the following steps:

[0111] Step a1: input physiological time series data into the first sub-feature extraction branch, and input EEG time series data into the second sub-feature extraction branch.

[0112] In step a2, the first sub-feature extraction branch extracts features from the physiological time series data to generate a physiological signal sequence.

[0113] The physiological signal sequence includes at least one of respiratory data features, electrocardiogram data time domain features, electrocardiogram data frequency domain features, electrocardiogram data nonlinear features, and electromyography data features.

[0114] Specifically, before extracting features from the physiological time series data, the physiological time series data may be preprocessed first.

[0115] Exemplary preprocessing steps for ECG and respiratory data include low-pass filtering, power frequency interference removal, and baseline drift elimination. Low-pass filtering primarily removes high-frequency interference from muscle signals, followed by a 50Hz notch filter to eliminate power frequency interference and, finally, baseline drift elimination. Electrodermal data preprocessing involves only low-pass filtering and power frequency interference removal.

[0116] Then, the first sub-feature extraction branch extracts features from the physiological time series data. The specific extraction process can be as follows:

[0117] ECG data and respiratory data are periodic. ECG data preprocessing also includes determining R-wave peaks. Adjacent R-wave peaks can be used to determine the duration of each heartbeat, and the inverse of the heartbeat time can be used to determine the heart rate. The R-wave peak detection method uses a threshold method. The principle of the threshold method is that the first sub-feature extraction branch first determines all peaks in the ECG data based on the peak function, and then determines the peak value greater than a certain threshold as the R-wave peak value. After determining the R-wave peak value, the heartbeat duration can be calculated. The instantaneous heart rate is then equal to the inverse of the heartbeat duration, and the average heart rate is equal to the average of all instantaneous heart rates.

[0118] Respiratory data preprocessing also includes determining respiratory peaks and troughs. The respiratory peak refers to the turning point where inhalation ends and exhalation begins, and the respiratory trough refers to the turning point where exhalation ends and inhalation begins. The determination of the respiratory peak and trough values ​​is based on the magnitude of the expiratory and inspiratory amplitudes. The principle of the respiratory peak and trough determination method is that, first, the first sub-feature extraction branch uses the peak function to determine all peaks and troughs of the respiratory data. Then, the expiratory amplitude and inspiratory amplitude are obtained, and for each expiratory amplitude and inspiratory amplitude, it is determined whether it is less than a threshold. For expiratory amplitudes and inspiratory amplitudes that are less than the threshold, the corresponding peaks and troughs should be deleted. Finally, the respiratory peak and trough values ​​are obtained.

[0119] The ECG data feature extraction process calculates the ECG data time domain features, frequency domain features and nonlinear features based on the RR interval data. The time domain features include 、 , SDNN, RMSSD, SDSD, pNN50, pNN20. Frequency domain features include TP, ULF, VLF, LF, HF, pLF, pHF, and LF / HF. Nonlinear features include the SD1 and SD2 values ​​of the Poincare scatter plot of RR interval data. The description of each feature is shown in Table 1.

[0120] Table 1 ECG data characteristics and their meanings

[0121]

[0122] To fully extract EMG data features, the diff function in the first sub-feature extraction branch obtains the first-order and second-order differences of the EDA data. Finally, the mean, median, standard deviation, minimum, and maximum values ​​are calculated for the EDA data, the first-order differences of the EDA data, and the second-order differences of the EDA data, for a total of 15 features. To fully extract EMG data features, the first sub-feature extraction branch first obtains respiratory peak, respiratory trough, expiratory amplitude, expiratory time, inspiratory amplitude, inspiratory time, respiratory time (determined by the interval between respiratory peaks), and respiratory time (determined by the interval between respiratory troughs). Then, the diff function in the first sub-feature extraction branch obtains the first-order and second-order differences of the respiratory data. Finally, the mean, median, standard deviation, minimum, and maximum values ​​are calculated for the respiratory data, the first-order differences of the respiratory data, and the second-order differences of the respiratory data, for a total of 120 features.

[0123] Finally, the first sub-feature extraction branch combines the obtained features to generate a physiological signal sequence.

[0124] Step a3: Based on the sparse attention scoring method, feature extraction is performed on the physiological signal sequence to generate a physiological signal matrix.

[0125] Specifically, the above step a3 may include the following steps:

[0126] Step a31 : linearly transform the physiological signal sequence to generate an initial query matrix, an initial key matrix, and an initial value matrix.

[0127] Specifically, the first sub-feature extraction branch can convert the physiological signal sequence into a vector representation that the model can process. Assuming that the length of the physiological signal sequence is T and the dimension of the physiological signal feature at each time step is d, the physiological signal sequence can be represented as a matrix X∈R T×d Then, the first sub-feature extraction branch can use linear transformation to map the input matrix X into the initial query matrix Q, the initial key matrix K and the initial value matrix V respectively. Specifically, through three learnable weight matrices Wq∈R d×dk , Wk∈R d×dk and Wv∈R d×dv Performing linear transformation, we get:

[0128] Q=XW q ∈R T×dk ;

[0129] K=XW k ∈R T×dk ;

[0130] V=XW v ∈R T×dv .

[0131] where dk and dv are the dimensions of the initial query matrix, initial key matrix, and initial value matrix, respectively.

[0132] Step a32 , calculating the similarity score between each query vector in the initial query matrix and all key vectors in the initial key matrix.

[0133] Specifically, the first sub-feature extraction branch can calculate each query vector q in the initial query matrix Q i and all key vectors k in the initial key matrix K j The similarity score S(q i ,k j ). Dot product similarity is usually used, that is, S(q i ,k j )=q i T k j .

[0134] Then, a softmax operation is performed on the similarity score to obtain the standard attention probability distribution. Assume that the formula is as follows: .

[0135] in, The purpose is to scale the dot product similarity to avoid the gradient vanishing or exploding problem.

[0136] Step a33 : introducing physiological signal specific priors and calculating the sparsity metric value corresponding to each query vector in the initial query matrix based on the similarity score.

[0137] Specifically, the first sub-feature extraction branch is for the i-th query vector q in the initial query matrix Q i , calculate its sparsity measure M(q i ). Assume the formula is as follows:

[0138] ;

[0139] in, is the query vector q i With all key vectors k j The similarity score S(q i ,k j ) performs a logarithmic form of the softmax operation, which measures the query vector q i Global correlation with the entire sequence. Is to calculate the query vector q i With all key vectors k j The average similarity score represents the query vector q i Average correlation with the series. , where HRV_mask(q i ) is the attention enhancement mask of the heart rate variability related channel. If the query vector q i Related to heart rate variability, the value of this mask is a positive number, otherwise it is 0. λ is the adaptive weight coefficient used to adjust the influence of this part.

[0140] Step a34: sort the query vectors according to the sparsity metric value, and select the top-u target query vectors with the highest sparsity metric value.

[0141] Specifically, the first sub-feature extraction branch sorts the query vectors based on the calculated sparsity metrics corresponding to each query vector and selects the top-u target query vectors with the highest sparsity metrics, where u = c·log(T) (c is an adjustable constant).

[0142] Step a35: compose a sparse query matrix based on the Top-u target query vectors.

[0143] Specifically, the first sub-feature extraction branch composes a sparse query matrix based on the Top-u target query vectors.

[0144] Step a36: Generate a physiological signal matrix based on the sparse query matrix, the initial key matrix and the initial value matrix.

[0145] Specifically, the electronic device 6 uses the filtered sparse query matrix to calculate the modified attention. The specific formula is as follows: .

[0146] in, is a sparse query matrix. In this way, only the sparse query matrix is ​​calculated The attention between the initial key matrix K, thus reducing the computational complexity from O(T 2 ) is reduced to O(uT).

[0147] Then, based on the modified attention, a physiological signal matrix is ​​generated.

[0148] Step a4: extract features from the physiological signal matrix based on a multi-cycle convolution kernel and output physiological modality features.

[0149] Specifically, assume that the physiological signal matrix is ​​X, and its dimension is [N, T], where N represents the number of signal channels and T represents the number of sampling points, that is, the length of the time series.

[0150] According to the characteristics of physiological signals and the purpose of analysis, a set of convolution kernels with different periods are designed. The size, shape and weight of these convolution kernels are usually learned through training. Assume that there are K convolution kernels, and the dimension of each convolution kernel is [N,L k ], where L k Represents the length of the kth convolution kernel, corresponding to different time periods. For example, a shorter convolution kernel may be used to capture high-frequency details in physiological signals, while a longer convolution kernel is used to extract low-frequency trends and periodic features.

[0151] Each convolution kernel is convolved with the physiological signal matrix X. For the kth convolution kernel, the convolution result Yk with X is calculated as follows: k =X×W k +b k . Where ∗ represents the convolution operation, W k is the weight matrix of the kth convolution kernel, b k Is the bias term. The result of the convolution operation Y k is a dimension of [N,TL k +1], which represents the characteristic response of the physiological signal after filtering by the kth convolution kernel.

[0152] The output feature matrix Y1, Y2, ..., Y of all convolution kernels KFusion. Common fusion methods include concatenation and summation. For example, through the concatenation operation, all feature matrices can be spliced ​​in the channel dimension to obtain a dimension of [N, (T-L1+1) + (T-L2+1) + ... + (TL K +1)], which is the physiological modality feature.

[0153] To better extract physiological modality features, subsequent processing can be performed on the fused feature matrix Y, such as pooling operations and the application of nonlinear activation functions. Pooling operations can reduce the dimensionality of features while retaining important feature information; nonlinear activation functions can increase the nonlinear expression capability of the model, making the extracted features more representative.

[0154] After the above steps, the final feature matrix is ​​the physiological modal feature. It contains the local and global features of the physiological signal at different time scales and can reflect the inherent patterns and regularities of the physiological signal.

[0155] In step a5, the second sub-feature extraction branch calculates the energy spectrum density and power spectrum density corresponding to the EEG time series data.

[0156] Specifically, electronic device 6 can select the power spectral density (PSD) and energy spectral density (ESD) corresponding to the EEG time series data as EEG analysis indicators. For dynamic signal analysis, since EEG time series data contains many frequency bands and is difficult to effectively divide in the time domain, frequency domain analysis is often used. Spectral analysis is typically used. This method extracts five frequency bands: [δ (1-4 Hz), θ (4-8 Hz), α (8-14 Hz), β (14-30 Hz), and γ (30-75 Hz). By arranging the harmonics decomposed from the original signal in descending order of energy, an energy spectrum can be formed, which represents the energy distribution of different frequency components in the signal.

[0157] Specifically, energy spectral density analysis is a method of EEG frequency domain analysis. In the process of analyzing dynamic signals, since EEG time series data often contains multiple frequency bands, it is very difficult to divide these frequency bands in detail in the time domain, so frequency domain analysis has become a common method. If the harmonics decomposed from the original signal are arranged according to their energy level, an energy spectrum is formed, which represents the energy distribution of different frequency components in the signal. The formula is as follows:

[0158] ;

[0159] in, is the energy spectral density, are the frequency components in the signal, is the Fourier transform of the signal, for The complex conjugate of .

[0160] PSD (power spectral density) is a measurement method that calculates the average power distribution ratio of a random variable per unit frequency, which is a form of mean square value. Its advantage is that it can convert the amplitude of the EEG time series signal that originally fluctuates with time into the power spectrum of the EEG time series signal that varies with frequency, thereby visualizing the distribution and transformation of the EEG time series rhythm. The second sub-feature extraction branch calculates the power spectral density of the EEG signal based on the pwelch function. It is based on the Welch method and estimates the PSD of the signal through the piecewise average periodogram method. This method selects the pwelch function and uses a Hamming window of 512 samples, an overlap of 256 samples, and a fast Fourier transform (FFT) of 1024 data points to estimate the frequency spectrum. Finally, the power spectrum is obtained by square the modulus of the spectrum, as shown in the formula:

[0161] ;

[0162] ;

[0163] in, is the power spectral density, is the sample size, is the Fourier transform result after windowing. is the energy in the frequency interval, f h is the upper limit of the frequency range, f i The lower limit of the frequency range.

[0164] Step a6: Extract features from the EEG time series data based on the energy spectral density and power spectral density, and output EEG modal features.

[0165] Specifically, the above step a6 may include the following steps:

[0166] Step a61: Map each frequency point in the EEG time series data to a first initial frequency band, a second initial frequency band, and a third initial frequency band.

[0167] Specifically, the electronic device 6 may map each frequency point in the EEG time series data to a first initial frequency band, a second initial frequency band, and a third initial frequency band. For example, the first initial frequency band is α (8-12 Hz), the second initial frequency band is β (13-30 Hz), and the third initial frequency band is γ (30-100 Hz).

[0168] Step a62: Calculate the energy mean and power mean corresponding to the first initial frequency band, the second initial frequency band, and the third initial frequency band respectively based on the energy spectrum density and the power spectrum density.

[0169] Specifically, based on the energy spectrum density and power spectrum density, the energy mean and power mean corresponding to the first initial frequency band, the second initial frequency band, and the third initial frequency band are calculated respectively to obtain the frequency domain feature X eeg ∈R T×B×Deeg (B=3 is the number of frequency bands).

[0170] Taking power spectrum density as an example, for the α frequency band, its power mean , F α is the number of frequency points within the alpha frequency band. These frequency band characteristics can reflect the intensity of brain electrical activity in different frequency ranges. In cognitive load research, power changes in different frequency bands are closely related to cognitive status. For example, increased beta wave power may be associated with high cognitive load.

[0171] Step a63 : Calculate weight information corresponding to the first initial frequency band, the second initial frequency band, and the third initial frequency band respectively based on the energy mean and power mean corresponding to the first initial frequency band, the second initial frequency band, and the third initial frequency band respectively.

[0172] Specifically, based on the fully connected layer, the energy mean and power mean corresponding to the first initial frequency band, the second initial frequency band and the third initial frequency band are processed to obtain the weight information α corresponding to the first initial frequency band, the second initial frequency band and the third initial frequency band respectively. f b ,S′(q,k b )=S(q,k b )•α f b .

[0173] For example, the electronic device 6 can convert the energy mean or power mean of each frequency band into a weight using a softmax function. Assuming that the weight is calculated using the energy mean, first calculate S = E1 + E2 + E3. Then, the weight w1 of the first initial frequency band is E1 / S, the weight w2 of the second initial frequency band is E2 / S, and the weight w3 of the third initial frequency band is E3 / S, and w1 + w2 + w3 = 1.

[0174] For example, if the power of the β band continues to increase over a period of time, after being processed by the fully connected layer and the Softmax function, the weight α of the β band will be f βThe time-frequency cross-attention technique uses a two-dimensional attention matrix and combines the distribution of energy spectral density or power spectral density at different time steps and frequency bands to capture the dominant frequency band at a specific time point. For example, during the peak period of a task, the energy or power of the gamma frequency band increases explosively. This technique can more accurately capture this change.

[0175] Step a64 : multiplying the first initial frequency band, the second initial frequency band, and the third initial frequency band by corresponding weight information respectively to generate a first target frequency band, a second target frequency band, and a third target frequency band.

[0176] Specifically, the first initial frequency band, the second initial frequency band, and the third initial frequency band are multiplied by corresponding weight information respectively to generate the first target frequency band, the second target frequency band, and the third target frequency band.

[0177] Step a65: extract the phase information corresponding to the EEG time series data through Hilbert transform to generate a phase matrix.

[0178] Specifically, the electronic device 6 uses the Hilbert kernel to convolve with the EEG time series data x(t) to obtain the signal y(t) after the Hilbert transform. From the frequency domain perspective, the Hilbert transform shifts the positive frequency components of the signal to the right by 90 and the negative frequency components to the left by 90. Based on this, the original EEG time series data x(t) and its Hilbert transformed signal y(t) can form an analytical signal z(t)=x(t)+jy(t). By analyzing the signal, the phase information can be calculated. In actual operation, by applying Hilbert transform to the EEG time series data after power frequency filtering and short-time Fourier transform, the phase information of each frequency point at different time points can be obtained, providing phase dimension data for constructing three-dimensional features.

[0179] Step a66, based on the first target frequency band, the second target frequency band, the third target frequency band and the phase matrix, integrates the information of the time dimension, frequency dimension and phase dimension corresponding to the EEG time series data, constructs the time-frequency-phase three-dimensional features, and generates EEG modal features.

[0180] Specifically, based on the first target frequency band, the second target frequency band, the third target frequency band and the phase matrix, the information of the time dimension, frequency dimension and phase dimension corresponding to the EEG time series data is integrated to construct the time-frequency-phase three-dimensional features.

[0181] Assume that the time step is T, the frequency dimension has three target frequency bands, namely the first target frequency band, the second target frequency band, and the third target frequency band, and the dimension of the phase matrix is ​​T×F (F is the number of frequency points). A three-dimensional tensor Feature∈R can be constructed T×3×2+T×F , where the first dimension represents time, the first three elements of the second dimension correspond to the energy spectrum density and power spectrum density information of the three target frequency bands, and the following elements correspond to the elements of the phase matrix. The specific representation is as follows:

[0182] ;

[0183] in, is the corresponding energy spectral density, is the corresponding power spectral density, is the phase matrix.

[0184] In step a7, the first sub-feature fusion network fuses the physiological modal features and the EEG modal features to generate temporal modal features.

[0185] Specifically, the first sub-feature fusion network performs splicing processing on the physiological modal features and the EEG modal features, or performs weighted fusion processing to generate temporal modal features.

[0186] Step S102: input the facial image data and the eye movement image data into a second feature extraction network to generate visual modality features.

[0187] Specifically, the second feature extraction network includes a third sub-feature extraction branch, a fourth sub-feature extraction branch, and a second sub-feature fusion network. The above step S102 may include the following steps:

[0188] Step b1: input the facial image data into the third sub-feature extraction branch, and input the eye movement image data into the fourth sub-feature extraction branch.

[0189] Step b2: The dynamic feature extraction branch in the third sub-feature extraction branch performs optical flow feature extraction on the facial image data, and outputs facial optical flow features corresponding to the facial image data.

[0190] Specifically, the facial image data consists of at least three frames, and the above step b2 may include the following steps:

[0191] Step b21: performing target detection on each frame of facial image data to determine a region of interest in each frame of facial image data.

[0192] The regions of interest include the orbicularis oculi, orbicularis oris, and frontalis muscles.

[0193] Specifically, the third sub-feature extraction branch may perform target detection on each frame of facial image data based on a target detection algorithm to determine a region of interest in each frame of facial image data.

[0194] Step b22 : Based on a preset light streaming algorithm, the motion vector corresponding to each facial image data is calculated according to the position information of each region of interest in the corresponding facial image data.

[0195] The motion vector includes horizontal displacement and vertical displacement.

[0196] The preset optical flow algorithm may be the Lucas-Kanade (LK) optical flow algorithm, the Farneback optical flow algorithm, or other optical flow algorithms. The embodiment of the present application does not specifically limit the preset optical flow algorithm.

[0197] Specifically, the third sub-feature extraction branch can extract two consecutive frames of facial images It, It+1∈R H×W×3 (RGB three channels), converted to a single-channel image Gt,Gt+1∈R H×W , eliminating color interference. Then, a 3×3 Gaussian kernel is used for noise reduction to reduce the impact of high-frequency noise on optical flow calculation.

[0198] Next, the third sub-feature extraction branch may calculate the motion vector corresponding to each facial image data based on the position information of each region of interest in the corresponding facial image data based on the Farneback optical flow algorithm.

[0199] Then, the motion vector is normalized to [-1, 1] and aligned with the original frame timestamp to form an optical flow feature sequence: Flowt=[Ut,Vt]∈R H×W×2 .

[0200] Step b23: Calculate the motion amplitude, motion direction, and motion acceleration corresponding to the motion vector.

[0201] Specifically, the third sub-feature extraction branch calculates the motion amplitude of the motion vector and direction of movement , as additional feature channels (a total of 4 channels: U, V, M, θ), enriching the representation dimension of motion features.

[0202] In addition, the third sub-feature extraction branch calculates the second-order difference ΔFlow of the motion vectors of three consecutive frames t =Flow t -Flow t-1 , capturing the motion acceleration corresponding to the motion vector and enhancing sensitivity to sudden changes in facial expressions (such as rapid opening and closing of eyelids when surprised).

[0203] Step b24: Fusing the motion vector, motion amplitude, motion direction, and motion acceleration to generate facial optical flow features.

[0204] Specifically, the motion vector, motion amplitude, motion direction and motion acceleration are spliced ​​to generate facial optical flow features.

[0205] Step b3: The static feature extraction branch in the third sub-feature extraction branch performs static feature extraction on the facial image data and outputs the facial static features corresponding to the facial image data.

[0206] Specifically, the static feature extraction branch includes multiple parallel branches; the above step b3 may include the following steps:

[0207] Step b31: The static feature extraction branch extracts features from the facial image data to obtain initial static features.

[0208] Specifically, the static feature extraction branch extracts features from facial image data based on the convolution kernel in the convolution layer to obtain initial static features.

[0209] For example, the convolution kernel in the convolution layer is 3×3 in size. This kernel is slid across the facial image data, performing a convolution operation on each local region. This operation multiplies and sums the corresponding elements to produce a new feature value. Using multiple convolution kernels, different image features, such as edges and textures, can be extracted to obtain initial static features.

[0210] Perform preliminary feature extraction on the input facial image data, converting the original facial image pixel information into a more representative feature map. This process can be seen as a simple feature abstraction of the image, preparing for subsequent multi-scale feature extraction.

[0211] Step b32: input the initial static features into each first parallel branch.

[0212] The initial static characteristics are then input to each first parallel branch.

[0213] In step b33, the first parallel branch weights the initial static features based on the channel and space dimensions to obtain weighted static features.

[0214] Specifically, the first parallel branch compresses the feature map of each channel into a single value through a global average pooling operation, obtaining global channel statistics. These statistics are then fed into a multi-layer perceptron (MLP), which learns to generate an appropriate weight for each channel based on these statistics. This weight can be viewed as a quantitative representation of the importance of the channel.

[0215] Furthermore, the first parallel branch can use convolution kernels to perform sliding convolution operations on the feature map, generating spatial weights by learning the kernel parameters. These convolution kernels can capture feature correlations within the local spatial region, thereby generating an appropriate weight value for each spatial location. Furthermore, attention mechanisms, such as the spatial attention module, can be combined to automatically learn the importance distribution of spatial locations.

[0216] Next, we perform channel-wise and spatial-dimensional weighting, respectively, to obtain channel weights and spatial weights. These two weights are then fused, for example, by combining them through simple multiplication or addition operations to obtain a comprehensive weight matrix. Finally, this weight matrix is ​​multiplied by the initial static features to obtain the weighted static features.

[0217] Step b34: performing local feature extraction on the weighted static features based on a preset number of multi-scale local feature extraction modules to obtain local static features at various scales.

[0218] Specifically, multi-scale parallel convolution is used for each multi-size local feature extraction module. By connecting two groups of 1x3 and 3x1 asymmetric convolution layers in series, the receptive field is expanded to a 5x5 feature area, and local feature extraction is performed on the weighted static features to obtain local static features at various scales.

[0219] Step b35: splicing the local static features to generate multi-scale local features.

[0220] Specifically, local static features at various scales are spliced ​​together to generate multi-scale local features.

[0221] For example, the extracted multi-scale initial features M(x) can be expressed as:

[0222] ;

[0223] in, 、 、 Represent convolution operations with different receptive field sizes, Represents the average pooling layer, which is used to extract features of different scales of the face. is the operation that connects these multi-scale features along the channel dimension. In addition, Use before and after the operation Convolution constructs a bottleneck structure to reduce the computational parameters of multi-scale feature extraction and ultimately generate multi-scale local features As shown below, that is: .

[0224] Step b36: shrink the multi-scale local features based on the global feature shrinkage attention module to generate target local features.

[0225] Specifically, the global feature shrinkage attention module first performs multi-scale local features Calculate the absolute value, and then obtain a new feature map through global pooling , to summarize and compress the global information of facial features. Finally, the feature map is simplified to a one-dimensional vector as shown in the following formula: .

[0226] in, Indicates the absolute value operation, W and H represent the width and height of the feature map respectively, A feature map with C channels will be generated, and the value in each channel represents the average value of all pixels in the feature map of the corresponding channel dimension.

[0227] Then the global information After two Convolution and sigmoid function to obtain a normalized scaling parameter As shown below, the parameter range is (0, 1) so that the threshold value obtained is not too large and is always positive, that is: .

[0228] in, is the scaling parameter, z is two layers The output of the convolution. In order to ensure that the threshold of the shrinkage function is positive and not too large to set a large number of features to zero, the threshold t can be expressed as: This approach allows different samples to obtain different thresholds so that the learned high-level features can become more discriminative.

[0229] After obtaining the threshold, the multi-scale local features The features with absolute values ​​lower than the threshold are eliminated, and the features with absolute values ​​greater than the threshold are shrunk toward 0. It can be expressed as follows: .

[0230] After shrinkage, currently irrelevant features can be set to zero, thereby alleviating the redundancy problem brought by multi-scale information and suppressing noise irrelevant to the facial emotion recognition task, while strengthening the relationship between local features and global features.

[0231] Finally, a cross-layer data path is used to connect the original features before multi-scale feature extraction with the output of the global feature shrinkage attention module, helping the network retain and reuse richer global features while preventing gradient disappearance and network degradation problems in deep neural networks. The method is shown in the following formula:

[0232] ;

[0233] in, is an expression feature map that combines local and global facial features, that is, the target local feature is the original facial feature map output by the previous layer, represents the local features extracted by the multi-scale local feature extraction module, Represents the facial features after being processed by the global feature shrinkage attention module.

[0234] Step b37: Input the target local features to the next parallel branch, and repeat this process until the last parallel branch is processed.

[0235] Specifically, the target local feature is input to the next parallel branch, and the cycle continues until the last parallel branch is processed. The processing process can be found above and will not be described in detail here.

[0236] In step b38, the outputs of each parallel branch are globally pooled, and facial static features are output based on the fully connected layer.

[0237] Specifically, the output results of each parallel branch are globally pooled, and facial static features are output based on the fully connected layer.

[0238] Step b4: The facial optical flow features and the facial static features are integrated to generate facial fusion features.

[0239] Specifically, the electronic device 6 may perform global average pooling on the motion amplitude M corresponding to the motion vector to obtain the motion significance vector S move ∈R C , characterizing the motion intensity of each channel feature.

[0240] Generate channel weights w through the fully connected layer move ∈(0,1) C , perform weighted fusion of facial static features and facial optical flow features to generate facial fusion features. The specific formula is as follows:

[0241] F final =wmove⊙Fstatic+(1-w move )⊙Fflow.

[0242] Among them, Fstatic is the facial optical flow feature, and Fflow is the facial static feature.

[0243] Step b5: The gaze point extraction branch in the fourth sub-feature extraction branch performs feature extraction on the eye movement image data and outputs the eye movement trajectory corresponding to the eye movement image data.

[0244] Specifically, the gaze point extraction branch identifies eye movement image data and determines eye key points within the image data. These key points are then tracked using a tracking algorithm (such as the KCF algorithm or MedianFlow algorithm). The tracking algorithm calculates eye movement information based on the positional changes of these key points between consecutive frames. During the tracking process, the position of these key points is recorded at each moment.

[0245] Then, according to the recorded position information of the key points of the eyeball, these points are connected to obtain the movement trajectory of the eyeball.

[0246] Step b6: extract features from the eye movement trajectory to determine the gaze point distribution features corresponding to the eye movement image data.

[0247] Specifically, feature extraction is performed on the eye movement trajectory to extract basic statistical features, spatial distribution features, time series features and motion features.

[0248] Specifically, the basic statistical features include: Number of fixations: This counts the total number of fixations in the eye movement image, reflecting the number of visual focus points during observation. Average fixation duration: This averages the duration of all fixations, reflecting the subject's attention to different content and information processing speed. Fixation density: This divides the image into several sub-regions and calculates the ratio of the number of fixations in each sub-region to the area of ​​that region to understand the density of fixations in different areas of the image.

[0249] The spatial distribution characteristics include: Central tendency: Calculate the mean of the gaze point coordinates to obtain the center position of the gaze point distribution, which can reflect the overall visual focus of the subject when observing the image. Discreteness: Measure the degree of dispersion of the gaze point around the center position by calculating the standard deviation or variance of the gaze point coordinates. The larger the standard deviation or variance, the more dispersed the gaze point is, and the less focused the subject's visual attention is. Spatial entropy: Used to describe the degree of disorder in the distribution of gaze points. The higher the spatial entropy value, the more uniform and random the distribution of the gaze points in space; the lower the entropy value, the more concentrated the gaze points are in certain specific areas.

[0250] Time series features include: Inter-fixation intervals: Calculating the intervals between adjacent fixations and analyzing their distribution can reveal the frequency and rhythm of the subject's visual attention shifts. Serial correlation: Analyzing the correlation of fixations in a time series by calculating autocorrelation or cross-correlation functions can determine whether there are periodic or trending changes.

[0251] Motion characteristics include: Movement speed: Calculating the movement speed of fixations based on the distance and time interval between adjacent fixations. Changes in speed can reflect the subject's visual search strategy and attention allocation during observation. Movement direction: Determining the direction of movement between each fixation point and counting the number of movements or the proportion of time in different directions can help analyze the subject's visual scanning patterns, such as whether there is a preferred scanning direction.

[0252] Then, principal component analysis is performed on the extracted features, converting the high-dimensional feature space into a low-dimensional space. The principal components that most significantly influence the gaze distribution characteristics are identified, achieving dimensionality reduction and visualization, more intuitively demonstrating the main patterns in gaze distribution. The extracted features are combined into a feature vector to generate a gaze distribution feature. This fully describes the gaze distribution characteristics corresponding to the eye movement image data, which can be used for subsequent classification, recognition, or correlation analysis with other data.

[0253] Step b7: the eye movement extraction branch in the fourth sub-feature extraction branch performs feature extraction on the eye movement image data, and outputs eye movement features corresponding to the eye movement image data.

[0254] Specifically, the eye movement extraction branch calculates the horizontal and vertical pupil movement speed based on changes in pupil position in continuous eye movement images. Pupil movement speed can reflect the subject's visual search speed and the speed of attention shifting. For example, pupil movement speed is generally faster when quickly browsing an image.

[0255] The eye movement extraction branch calculates the eye rotation angle based on the relative position changes of the pupil and iris, as well as the motion information of feature points around the eye. This eye rotation angle can help analyze changes in the subject's gaze direction and visual focus, and is important for understanding the subject's visual cognitive process.

[0256] The eye movement extraction branch detects changes in eye closure in eye movement images and counts the number of blinks per unit time, known as blink frequency. Blink frequency can reflect the subject's level of fatigue and concentration. Generally speaking, blink frequency increases when fatigue or inattention occurs.

[0257] Then, pupil movement velocity, eye rotation angle, and blink frequency are fused to form a comprehensive eye movement feature vector. Simple concatenation methods can be used to connect different types of feature vectors, or more complex fusion algorithms, such as principal component analysis-based feature fusion or neural network-based feature fusion, can be used to reduce feature dimensionality, remove correlation between features, and improve feature representativeness and robustness.

[0258] Step b8: integrating the gaze point distribution features and the eye movement features to generate eye movement fusion features.

[0259] Specifically, the fourth sub-feature extraction branch combines the gaze distribution features and eye movement features to generate eye movement fusion features. This can be achieved by combining the gaze distribution features and eye movement features using a simple concatenation method, or by using more complex fusion algorithms, such as principal component analysis-based feature fusion or neural network-based feature fusion, to reduce feature dimensionality, remove correlation between features, and improve feature representativeness and robustness.

[0260] Step b9: fusing the facial fusion features and the eye movement fusion features to generate visual modality features.

[0261] Specifically, facial fusion features and eye movement fusion features are fused to generate visual modal features. Simple concatenation methods can be used to connect facial fusion features and eye movement fusion features. Alternatively, more complex fusion algorithms, such as principal component analysis-based feature fusion and neural network-based feature fusion, can be used to reduce feature dimensionality, remove correlation between features, and improve feature representativeness and robustness.

[0262] Step S103: fusing the temporal modal features and the visual modal features to generate target fused features.

[0263] In an optional implementation of the present application, the above step S103 may include the following steps:

[0264] Step S1031: Use the visual modality features as the first query matrix, and the temporal modality features as the first key matrix and the first value matrix.

[0265] Specifically, the electronic device 6 uses the visual modality feature as the first query matrix and the temporal modality feature as the first key matrix and the first value matrix.

[0266] Step S1032: Calculate the first dependency weight of the visual modality feature on the temporal modality feature.

[0267] Specifically, the similarity between the visual modality features and the temporal modality features is measured by calculating the dot product of the first query matrix Q and the first key matrix K. The specific calculation formula is: sim=Q•KT. i,j represents the similarity score between the i-th visual modality feature and the j-th temporal modality feature.

[0268] In order to avoid gradient instability caused by excessive dot product results, the similarity score is usually scaled. The scaling factor is ,Right now: .

[0269] Then, the Softmax function is applied to the scaled similarity scores to convert them into probability distributions, thereby obtaining the first dependency weight matrix W. The calculation formula of the Softmax function is: .

[0270] Step S1033: Use the temporal modal features as the second query matrix, and the visual modal features as the second key matrix and the second value matrix.

[0271] Specifically, the temporal modality features are used as the second query matrix, and the visual modality features are used as the second key matrix and the second value matrix.

[0272] Step S1034: Calculate the second dependency weight of the temporal modal feature on the visual modal feature.

[0273] Specifically, referring to the calculation method of the first dependency weight above, the second dependency weight is calculated and obtained, which will not be described in detail here.

[0274] Step S1035 , multiplying the time series modal feature by the first dependency weight to obtain the target time series feature.

[0275] Specifically, the electronic device 6 multiplies the time series modal feature by the first dependency weight to obtain the target time series feature.

[0276] Step S1036: Multiply the second dependency weight by the visual modality feature to obtain the target visual feature.

[0277] Specifically, the electronic device 6 multiplies the second dependency weight by the visual modality feature to obtain the target visual feature.

[0278] Step S1037: Merge the target temporal features and the target visual features to generate target fusion features.

[0279] Specifically, the electronic device 6 combines the target temporal features and the target visual features to generate target fusion features.

[0280] Step S104: outputting the current workload level corresponding to the target user based on the target fusion feature.

[0281] Specifically, the preset workload identification model outputs the current workload level corresponding to the target pilot based on the target fusion features.

[0282] The flight training system provided in the embodiment of the present application inputs physiological time series data and EEG time series data into a first feature extraction network to generate time series modal features. This ensures the accuracy of the generated time series modal features. Facial image data and eye movement image data are input into a second feature extraction network to generate visual modal features, thereby ensuring the accuracy of the generated visual modal features. In addition, inputting data into different networks according to modality avoids the increased complexity caused by the mixing of multiple types of data. The characteristics and processing methods of different modal data vary greatly. Separate processing allows each network to focus on feature extraction of a specific type of data, reducing the burden of network training and calculation, and improving data processing efficiency. At the same time, mutual interference between different modal data is reduced, making feature extraction more accurate and efficient. The visual modal features are used as the first query matrix, and the time series modal features are used as the first key matrix and the first value matrix; the first dependency weight of the visual modal features on the time series modal features is calculated, ensuring the accuracy of the calculated first dependency weight. The temporal modal features serve as the second query matrix, and the visual modal features serve as the second key matrix and second value matrix. The second dependency weight of the temporal modal features on the visual modal features is calculated, ensuring the accuracy of the calculated second dependency weight. The temporal modal features are multiplied by the first dependency weight to obtain the target temporal features, and the visual modal features are multiplied by the second dependency weight to obtain the target visual features. This weighting operation strengthens the components with high correlation with the other modal features, allowing the target features to better reflect the relationship between the different modalities. Furthermore, this weighting operation weakens the components with low correlation with the other modal features, reducing redundant information in the features and improving their quality and efficiency. The target temporal features and target visual features are merged to generate a target fused feature. This achieves a deep fusion of visual and temporal modalities. The fused feature integrates the advantages of both modalities and can more comprehensively and accurately reflect the pilot's working state. For example, by combining visual information such as facial expressions and eye movements with temporal information such as physiological signals and EEG signals, it can assess the pilot's cognitive load, fatigue, and other aspects from multiple perspectives. Based on the target fusion features, the current workload level corresponding to the target user is output. The target fusion features incorporate multiple aspects of information, more accurately reflecting the pilot's actual workload. Compared to assessments based solely on single modal features, this approach reduces the probability of misjudgments and missed detections, providing more reliable workload assessment results for pilots, flight management personnel, and related systems.

[0283] In an optional embodiment of the present application, Figure 5 As shown, the above-mentioned “generating a backup virtual driving scenario based on preset training instructions, operation action data, button action data, and parameter setting data” may include the following steps:

[0284] Step S201: Generate an initial virtual driving scene based on preset training instructions.

[0285] In an optional implementation, the above step S201 may include the following steps:

[0286] Step S2011: perform semantic recognition on the preset training instructions and extract key information from the preset training instructions.

[0287] The key information includes at least one of the flight mission type, flight area, and weather conditions.

[0288] Specifically, the electronic device 6 performs preliminary cleaning on the preset training instruction text to remove some meaningless characters, punctuation marks, spaces, etc. For example, the extra spaces and line breaks in the instruction are removed to make the text more concise.

[0289] Next, the continuous text sequence is segmented into meaningful words or phrases. For Chinese commands, tools like Jieba Word Segmenter can be used; for English commands, simple word segmentation based on spaces can be used. For example, "In thunderstorm weather, perform air patrol missions in North China" would be segmented into "in," "thunderstorm weather," "under," "in," "North China," "perform," and "air patrol mission."

[0290] Next, each segmented word is labeled with its part of speech, such as noun, verb, or adjective. This facilitates subsequent syntactic analysis and semantic understanding. For example, "thunderstorm" is labeled as a noun, and "execute" is labeled as a verb. The entity types to be identified are clearly defined. In flight training scenarios, these primarily include flight mission type, flight area, and weather conditions.

[0291] Finally, electronic device 6 uses machine learning or deep learning methods to train a named entity recognition model. Common methods include conditional random fields (CRFs), long short-term memory (LSTM) combined with conditional random fields (LSTM-CRFs), and fine-tuning based on pre-trained language models (such as BERT).

[0292] The preprocessed text is fed into a trained named entity recognition model, which then outputs the type and location of each entity in the text. For example, given the instruction "Conduct an aerial mapping mission in the western region under cloudy weather," the model will identify "cloudy weather" as the weather condition, "western region" as the flight area, and "aerial mapping mission" as the flight mission type.

[0293] Furthermore, electronic device 6 analyzes the grammatical structure of the sentence and determines the dependency relationships between words. For example, using a dependency syntax analysis tool, it identifies a subject-verb relationship between "execute" and "air patrol mission." Combining the results of named entity recognition and syntax analysis, it further understands the semantics of the sentence. For example, based on the words in the instruction and the relationships between them, it determines that the core semantics of the instruction is to execute a specific flight mission under specific weather conditions and flight areas.

[0294] Finally, based on the previous analysis results, electronic device 6 filters out key information from the text, such as the flight mission type, flight area, and weather conditions. If the instruction does not explicitly mention certain types of key information, the corresponding information will be blank. For example, in the instruction "Conduct a low-altitude flight mission," the flight mission type is "low-altitude flight mission," and the flight area and weather conditions information are blank.

[0295] In step S2012, each key information is used as an initial node, the dependency relationship between each key information is used as an initial edge, and attribute information is set for each initial node to construct an initial graph structure.

[0296] Specifically, key information extracted from pre-set training instructions, such as the mission type, flight area, and weather conditions, serves as the initial node. For example, the mission type might be "air patrol" or "aerial mapping"; the flight area might be "North China" or "Western China"; and the weather conditions might be "thunderstorm" or "cloudy." Each specific key information item serves as a separate initial node in the graph structure.

[0297] Then, initial edges are constructed based on the dependencies between key information. For example, different flight mission types may have specific requirements or restrictions on flight areas. For example, maritime patrol missions are typically conducted in coastal areas or specific sea areas. Therefore, a dependency exists between the "Maritime Patrol" flight mission type node and the corresponding coastal flight area node, and an edge can be set to connect them. Certain flight missions can only be conducted under specific weather conditions, or certain weather conditions may affect flight missions. For example, "heavy fog" weather conditions may restrict certain flight missions requiring visual operation, such as "low-altitude flight." In this case, a dependency exists between the "heavy fog" weather condition node and the "low-altitude flight" flight mission type node, and this relationship is represented by an edge. Different flight areas may have their own common weather conditions. For example, flights in tropical regions may be more prone to weather conditions such as heavy rain and typhoons. Therefore, a dependency exists between the "tropical region" flight area node and weather condition nodes such as "heavy rain" and "typhoon," and these can be connected by edges.

[0298] Next, set the attribute information for each initial node.

[0299] The flight mission type node attributes include: difficulty level, required flight skills, mission objectives, etc. For example, the "Aerial Refueling" mission has a high difficulty level and requires pilots to have precise operating skills and extensive experience. The mission objective is to refuel other aircraft.

[0300] Flight Area Node Attributes: You can set attributes such as the area's geographic features, airspace restrictions, and navigation facilities. For example, a "mountainous" flight area might have complex terrain, including mountains and canyons; airspace restrictions might include altitude restrictions and military restricted areas; and navigation facilities might be relatively scarce, requiring pilots to rely on other methods for navigation.

[0301] Weather Condition Node Properties: You can set weather condition attributes such as intensity, duration, and impact on flight. For example, a thunderstorm can have varying levels of intensity, such as weak, moderate, or strong, and can last from a few hours to several days. This can have a significant impact on flight safety, potentially causing turbulence and communication interruptions.

[0302] All initial nodes (flight mission type, flight area, and weather conditions) are connected by edges based on their dependencies, forming a graph structure. In this graph, initial nodes represent key information, edges represent the dependencies between key information, and node attributes further enrich the description of key information. For example, for the sentence "Perform an air patrol mission in North China during thunderstorms," ​​a graph structure can be constructed in which the "Thunderstorm" node is connected to the "North China" node by an edge, the "Thunderstorm" node is connected to the "Air Patrol" node by an edge, and the "North China" node is connected to the "Air Patrol" node by an edge. Each node also has corresponding attribute information to describe its specific characteristics. This initial graph structure more clearly represents the relationships between key information in the pre-set training instructions, providing a foundation for subsequent analysis and processing.

[0303] Step S2013: Perform convolution on the initial graph structure to obtain initial node features corresponding to each initial node in the graph structure.

[0304] Specifically, before performing convolution operations, it is necessary to clarify the mathematical representation of the initial graph structure. Generally speaking, the graph G = (V, E) consists of the initial node set V and the edge set E. For each initial node vi∈V, there is a corresponding eigenvector xi. The eigenvectors of all initial nodes are combined to form the initial node feature matrix X∈R N×D , where N is the number of initial nodes and D is the dimension of the initial node features. At the same time, the connection relationship of the initial graph structure can be expressed using the adjacency matrix A∈R N×NTo indicate that if there is an edge between the initial node i and the initial node j, then Aij=1, otherwise Aij=0.

[0305] The operation steps of graph convolution operation include the following:

[0306] 1. Preprocessing of the adjacency matrix: In order to better aggregate the information of adjacent nodes, the adjacency matrix is ​​usually preprocessed. A common method is to normalize the adjacency matrix, such as calculating the symmetric normalized Laplace matrix:

[0307] ;

[0308] ;

[0309] .

[0310] Among them I N is the N×N identity matrix, yes The degree matrix of is the normalized adjacency matrix.

[0311] Convolution operation process: Suppose we have a graph convolution layer whose input is the node feature matrix X, and the weight matrix of this layer is (D′ is the dimension of the output feature.) The formula for the graph convolution operation is as follows: . Among them, σ is the activation function, such as ReLU (RectifiedLinearUnit) function, H∈R N×D′ is the new node feature matrix obtained after the convolution operation. In this operation process, The aggregation of node features is achieved, that is, each initial node will comprehensively consider the feature information of its adjacent nodes; multiplying by the weight matrix W performs a linear transformation on the aggregated features; finally, nonlinearity is introduced through the activation function to obtain the updated node feature representation.

[0312] Optionally, to further extract higher-level node features, multiple graph convolutional layers are typically stacked. The output of each layer serves as the input to the next layer. Through multiple layers of convolution, nodes can capture information about nodes at greater distances, thereby mining more complex spatial connections within the graph structure. For example, after the first layer of graph convolution, node features initially aggregate information about directly adjacent nodes; after the second layer of graph convolution, node features incorporate information about adjacent nodes of adjacent nodes, and so on.

[0313] Step S2014: generating an initial virtual driving scene based on the features of each initial node.

[0314] Specifically, the electronic device 6 may search the storage space for an initial virtual driving scene that matches the initial node characteristics according to the characteristics of each initial node.

[0315] Step S202 : Based on the initial virtual driving scene, a backup virtual driving scene is generated based on the operation action data, button action data, and parameter setting data.

[0316] Specifically, the above step S202 may include the following steps:

[0317] Step S2021 : abstracting the operation action data, button action data, and parameter setting data into update nodes, and establishing update edge connections with each initial node in the initial graph structure.

[0318] Specifically, the operation action data contains information such as the execution order corresponding to each operation action and the operation force corresponding to each operation action. For these data, they are first parsed and classified. For example, specific operation actions such as "pulling the stick to take off", "adjusting the flap angle", and "stepping on the brakes" are regarded as independent operation action types. Then, an update node is created for each operation action type, and the node records the relevant attributes of the operation action, such as the name of the operation action, the operation stage to which it belongs (take-off, cruise, landing, etc.), the range of the operation force, etc. For example, the "pulling the stick to take off" node can record its name as "pulling the stick to take off", the operation stage as "take-off", and the range of the operation force may be within a certain angle range. Through this abstract method, specific operation actions are converted into nodes in the graph structure, which is convenient for subsequent analysis and processing.

[0319] The button action data is mainly the button action information input by the target user on the button control panel 3 based on the preset training instructions. Similarly, these button actions are classified and organized. For example, button actions such as "autopilot mode on", "navigation system setting confirmation", and "light switch operation" are abstracted into corresponding update nodes. Each button action update node records the button's name, function description, and the system or device being operated. Taking the "autopilot mode on" button action node as an example, its name can be recorded as "autopilot mode on", the function description can be used to turn on the aircraft's autopilot system, and the device being operated is the aircraft's autopilot control system, etc., thereby converting the button action data into valid nodes in the graph structure.

[0320] Parameter setting data is the data set by the target user for flight parameters, such as flight altitude, speed, heading, engine power and other parameter settings. These parameter setting data are classified according to the type of parameter, and each parameter type can be abstracted as an update node. For example, update nodes such as "flight altitude setting", "speed setting" and "engine power setting". Each node records the name of the parameter, the set numerical range, the impact on the flight and other attributes. For example, the "flight altitude setting" node can record its name as "flight altitude setting", and the numerical range is determined according to the actual flight situation. The impact on the flight includes information such as the impact on the aircraft's aerodynamic performance and fuel consumption, thereby realizing the conversion of parameter setting data to update nodes.

[0321] When establishing update edge connections, electronic device 6 needs to determine the associations between the operation action data, button action data, parameter setting data, and each initial node in the initial graph structure (e.g., flight mission type, flight area, weather conditions, etc.) as the connection basis. For example, the operation action "pull the stick to take off" is directly associated with the flight mission type "takeoff training" because "pull the stick to take off" is a specific operation action in the "takeoff training" mission; the button action "autopilot mode on" may be related to the flight mission type "cruise mission" because the autopilot mode is often used during the cruise phase; the parameter setting "flight altitude setting" may be related to the flight area and weather conditions, and different flight areas and weather conditions may have different requirements and restrictions on flight altitude.

[0322] Then, electronic device 6 establishes an update edge connection between the update node and the initial node based on the determined association. If an operation action is associated with a certain flight mission type, an edge is added between the corresponding operation action update node and the flight mission type initial node. If a button action is related to flight operations under certain weather conditions, an edge connection is established between the button action update node and the corresponding weather condition initial node. For parameter setting data, if it is related to both flight area and flight mission type, an edge connection is established between the parameter setting update node and the flight area initial node and the flight mission type initial node, respectively. The direction of the edge can be determined based on the specific association logic. For example, pointing from the operation action node to the flight mission type node indicates that the operation action is intended to complete a specific flight mission; pointing from the parameter setting node to the flight area node indicates that the parameter setting is specific to a specific flight area.

[0323] To more accurately describe the relationship between the update node and the initial node, electronic device 6 assigns corresponding attributes to each update edge. Attributes may include the strength of the association, the degree of influence, and so on. For example, for the edge between the "pull the stick to take off" action node and the "takeoff training" flight mission type node, the strength of the association may be set to high, as "pull the stick to take off" is a critical action in the "takeoff training" mission. For the edge between the "autopilot mode on" button action node and the "cruise mission" flight mission type node, the degree of influence may be set to medium, indicating that the autopilot mode plays a role in the cruise mission but is not the only operating method. By setting edge attributes, the graph structure can more clearly reflect the complex relationships between nodes, providing richer information for subsequent analysis and processing.

[0324] Step S2022, according to the association relationship between the operation action data, button action data, parameter setting data and the preset training instructions, determine the weight and direction of each updated edge, and update the attribute information of each updated node to generate a target graph structure.

[0325] Specifically, electronic device 6 determines the inherent relationship between operation action data (e.g., operation sequence and force), button action data (button operation status), parameter setting data (flight parameter settings), and preset training instructions. For example, if the preset training instruction is to execute a takeoff mission, then the operation action data such as pushing the throttle and pulling the lever are closely related to it.

[0326] The weight of the update edge is determined based on the closeness of the aforementioned association. If a certain action is a key step in completing a training instruction and is frequently executed, the weight of the update edge between it and the relevant node will be high. For example, in takeoff training, the edge weight between the "throttle push" action and the "takeoff training" node will be large.

[0327] The direction of the update edge is determined based on the causal or impact relationship between the data. Operations are usually performed to implement training instructions, so the edge of the operation update node generally points to the initial node of the flight mission type. For example, the edge of the "Pull Rod" action node points to the "Takeoff Training" node.

[0328] Update the properties of nodes such as operation actions, button actions, and parameter settings based on their association with preset training instructions. For example, for operation action nodes, update the execution conditions and expected effects; for button action nodes, update the function description and applicable scenarios; and for parameter setting nodes, update the value range and impact on flight performance.

[0329] The determined updated edge weights and directions, as well as the updated node attribute information, are integrated into the graph to form the target graph structure. This structure clearly presents the relationship between various data types and preset training instructions, providing a more accurate foundation for subsequent flight training analysis and virtual scene generation.

[0330] Step S2023: Perform convolution on the target graph structure to obtain target node features corresponding to each target node in the graph structure.

[0331] The target nodes include initial nodes and update nodes.

[0332] Specifically, during the computation, the adjacency matrix for each target node is preprocessed, such as normalization, to better aggregate neighboring node information. Electronic device 6 then computes the target node feature matrix corresponding to each target node with the processed adjacency matrix and weight matrix, and introduces nonlinearity through an activation function to obtain a new target node feature representation. During this process, each target node comprehensively considers the influence of its own characteristics and the characteristics of its surrounding neighboring nodes.

[0333] After the convolution operation, each target node in the target graph (including initial and updated nodes) receives an updated feature representation, known as the target node feature. These features combine the node's original attribute information with relevant information transmitted from connected nodes, more comprehensively reflecting the node's role and context within the entire graph. For example, the target node feature of an action update node not only contains information about the action itself but also incorporates information related to the flight mission, region, and other aspects of the action. This provides a richer and more valuable data foundation for subsequent analysis and applications based on these features, such as generating virtual driving scenarios.

[0334] In step S2024, each target node feature matrix is ​​input into a preset recurrent neural network at each time step to obtain the current target state feature.

[0335] Specifically, the target node feature matrix is ​​passed as input to the input layer of the preset recurrent neural network. At the same time, the state of the hidden layer is usually initialized to a zero vector or random value.

[0336] For each time step t, the input layer converts the target node feature matrix X at the current time step into t Passed to the hidden layer. The hidden layer will be based on the current input X t and the hidden state ht-1 of the previous time step to calculate the hidden state h of the current time step t The specific calculation process is as follows:

[0337] For ordinary RNN, the calculation method is h t =σ(W xh X t +W hh h t-1 +b h ), where σ is the activation function, such as tanh or ReLU, and W xh and Whh is the weight matrix, b h is the bias term.

[0338] For LSTM, the calculation process is more complicated, including the input gate i t 、Forget Gate t , output gate o t and cell state c t The specific formula is:

[0339] i t =σ(W xi X t +W hi h t-1 +b i )

[0340] f t =σ(W xf X t +W hf h t-1 +b f )

[0341] c t =f t ⊙c t-1 +i t ⊙tanh(W xc X t +W hc h t-1 +b c )

[0342] o t =σ(W xo X t +W ho h t-1 +b o )

[0343] h t =ot⊙tanh(c t )

[0344] The calculation method of GRU is similar to LSTM, but it is relatively simple. It combines the input gate and forget gate into an update gate and directly updates the hidden state.

[0345] After the hidden layer calculates the hidden state ht for the current time step, it passes it to the output layer. The output layer calculates the output yt based on the hidden state ht, the output weight matrix Why, and the bias term by: yt = Whyht + by. In some cases, the output may be the current target state feature; in other cases, further processing or transformation of the output may be required to obtain the current target state feature.

[0346] Step S2025 : Based on the current target state characteristics, the initial virtual driving scene is adjusted to generate a backup virtual driving scene.

[0347] Specifically, the current target state features are associated with various elements in the initial virtual driving scene. For example, if a feature indicates that the trainee's aircraft speed control is unstable during operation, this information is correlated with aircraft speed-related elements in the scene (such as the instrument panel speed display and speed control devices) and flight environment elements (such as wind speed, airflow, and other factors affecting speed). Based on the analysis results, adjustments to the initial virtual driving scene are made. If the current operation is found to be inconsistent with the flight mission requirements, the mission prompt information and target location in the scene may need to be modified. If the current weather conditions do not match the trainee's performance, the weather conditions can be adjusted appropriately, such as changing from sunny to cloudy to increase the flight difficulty. For features related to parameter settings, if certain parameter settings are found to be unreasonable, the aircraft parameters in the scene, such as engine power and flap angle, are adjusted accordingly. Based on the adjustment decisions, the initial virtual driving scene is modified and updated. This includes adjusting the position, posture, and attributes of 3D models in the scene (such as aircraft and buildings); modifying environmental effects (such as lighting and weather effects); and optimizing interactive elements (such as instrument panel displays and button feedback). Through these adjustments, alternative virtual driving scenarios are generated that better reflect the current training status and meet training needs, providing trainees with a more targeted and effective training environment.

[0348] In the flight training system provided by the embodiments of the present application, electronic device 6 performs semantic recognition on preset training instructions and extracts key information from them, ensuring the accuracy of the extracted key information. This key information clarifies the core direction for subsequent scenario construction, ensuring that the generated initial virtual driving scenario is highly consistent with the training objectives, and avoiding deviations in scenario construction and waste of resources. An initial graph structure is constructed, with each key information item as an initial node, the dependencies between each key information item as initial edges, and attribute information assigned to each initial node. This initial graph structure clearly displays the connections between each key information item. Using this initial graph structure, electronic device 6 can systematically sort out these complex relationships, ensuring that each element in the scenario is coordinated and logically consistent when generating the initial virtual driving scenario. A convolution operation is performed on the initial graph structure to obtain initial node features corresponding to each initial node in the graph structure. This effectively mines the feature information of each initial node in the graph. The convolution operation aggregates information from a node and its adjacent nodes to generate more representative initial node features for each node. These features not only contain the node's own key information but also incorporate comprehensive information from its associations with other nodes. Based on these initial node features, an initial virtual driving scenario is generated. This feature-based scenario generation method enriches the initial virtual driving scenario with richer details and greater realism, providing a more comprehensive and in-depth training environment for pilots, helping them better understand and master the key operational aspects of different flight missions under various conditions. Electronic device 6 abstracts the operation action data, button action data, and parameter setting data into update nodes and establishes update edges connecting them to each initial node in the initial graph structure. This operation breaks the relatively static nature of the initial scenario construction and incorporates the pilot's real-time operation data during training into the graph structure. This dynamic data fusion method eliminates the virtual driving scenario from being limited to a fixed pattern based on preset instructions, allowing it to adjust in real time with the pilot's operation. This enhances the scenario's adaptability to the training process, making training more closely aligned with the ever-changing operational scenarios of real flight and improving the pilot's adaptability in actual operations. Based on the correlation between the operation action data, button action data, parameter setting data, and the preset training instructions, the weight and direction of each update edge are determined, and the attribute information of each updated node is updated to generate the target graph structure. The weight setting reflects the closeness of the relationship between the data, while the direction clarifies the logic of information transmission. In this way, the target graph structure can more accurately organize the logical relationships between various types of information during training, avoiding unreasonable element combinations or operational conflicts in the scenario. When generating alternative virtual driving scenarios, based on the optimized target graph structure, the elements and operational processes in the scenario will be more consistent with actual flight logic, further enhancing the scene's realism and credibility, and helping pilots develop correct flight operation logic. Convolution operations are performed on the target graph structure to obtain the target node features corresponding to each target node in the graph structure.Convolution operations fully exploit the deep features of each node and its associated information within the target graph structure. Compared to feature extraction from the initial graph structure, this operation incorporates real-time operational data, resulting in richer and more comprehensive features. Based on these more detailed and dynamic target node features, the generated backup virtual driving scenarios can more precisely depict details such as aircraft motion, instrument displays, and environmental changes. This allows the scenarios to more realistically simulate the complexities of actual flight, providing a more immersive training environment and helping pilots gain a deeper understanding of the relationship between operational and flight conditions. Each target node feature matrix is ​​input into a pre-set recurrent neural network at each time step to obtain the current target state features. Based on these current target state features, the initial virtual driving scenario is adjusted to generate the backup virtual driving scenario. Recurrent neural networks excel at processing time series data and can analyze dynamic trends and patterns in data during training. Based on these analysis results, the system can make targeted adjustments to the initial virtual driving scenario, such as pre-generating scenario elements relevant to the predicted operation or adjusting the scenario's difficulty. For experienced pilots, complex weather conditions or equipment failures can be added as appropriate; for less experienced pilots, more operational prompts and auxiliary information can be provided. This precise scene adjustment mechanism enables the backup virtual driving scenes to be dynamically adapted according to the pilot's real-time performance and training needs, achieving personalized training, improving the targetedness and effectiveness of training, and helping pilots improve their flying skills faster.

[0349] In an optional embodiment of the present application, Figure 6 As shown, the above-mentioned “adjusting the complexity of the backup virtual driving scenarios according to the target cognitive load level to generate the target virtual driving scenario” may include the following steps:

[0350] Step S301 : inputting physiological time series data, EEG time series data, eye movement image data, facial image data, and target cognitive load level into a meta-learner.

[0351] Specifically, the electronic device 6 inputs the physiological time series data, the electroencephalogram time series data, the eye movement image data, the facial image data, and the target cognitive load level into the meta-learner.

[0352] Step S302: The meta-learner obtains an initial cognitive load-scenario response model.

[0353] Among them, the initial cognitive load-scenario response model includes an input layer, a hidden layer, and an output layer; the input layer is responsible for receiving data, and its number of neurons is set according to the data feature dimension; the hidden layer is composed of multiple layers of neurons, which processes the data through complex connection weights; and the output layer outputs the final result.

[0354] Specifically, the meta-learner searches for an initial cognitive load-scenario response model from the storage space, wherein the initial cognitive load-scenario response model is trained based on the training data.

[0355] In step S303 , the meta-learner adjusts the initial cognitive load-scenario response model based on the physiological time series data, the EEG time series data, the eye movement image data, and the facial image data to generate a target cognitive load-scenario response model corresponding to the target user.

[0356] Specifically, the meta-learner first extracts features from the input physiological time-series data, EEG time-series data, eye movement image data, and facial image data, and then fuses these features to generate a comprehensive feature vector. This fusion can be done through simple concatenation or weighted combination using a specific algorithm to form a comprehensive feature vector. This comprehensive feature vector incorporates information on the user's physiology, psychology, and behavior, providing a more comprehensive reflection of their cognitive state.

[0357] Based on the comprehensive feature vector, the meta-learner analyzes the target user's unique characteristics and patterns. It then adjusts parameters such as the connection weights of neurons in the model to better align the model's output with the relationship between the target user's cognitive load and the scenario. This adjustment can be done using a gradient descent algorithm, which calculates the error between the model output and the true value and backpropagates the gradient to update the connection weights. Furthermore, the meta-learner can utilize optimization techniques, such as adaptive learning rate adjustment and regularization, to improve the model's training efficiency and generalization capabilities.

[0358] The meta-learner doesn't adjust the model all at once; instead, it continuously optimizes the model through multiple iterations. In each iteration, the meta-learner adjusts the model's parameters based on new input data and the model's output, gradually adapting the model to the characteristics of the target user. As the number of iterations increases, the model becomes increasingly accurate in matching the target user's cognitive load and context.

[0359] The target cognitive load-scenario response model generated after adjustment by the meta-learner can more accurately output appropriate results based on the state of the target user.

[0360] Step S304: input the target cognitive load level into the target cognitive load-scenario response model, and output a scenario complexity adjustment parameter.

[0361] Among them, the scenario complexity adjustment parameters include fault type adjustment, fault frequency adjustment, and environment complexity adjustment.

[0362] Specifically, the target cognitive load level is input into the target cognitive load-scenario response model. After receiving the target cognitive load level, the model's internal layers (including the input layer, hidden layer, and output layer) perform a series of calculations and analyses on the input data based on learned parameters and connection weights. The hidden layer extracts features from the input data and processes these features through complex operations to uncover the potential connection between cognitive load level and scenario complexity.

[0363] After processing and analysis by the model, the output layer finally outputs the scene complexity adjustment parameters.

[0364] Step S305 : performing complexity adjustment on the backup initial virtual driving scene based on the scene complexity adjustment parameter to generate a target virtual driving scene.

[0365] Specifically, the electronic device 6 can adjust parameters based on the fault type, adding or replacing corresponding fault elements in the backup virtual flight scenario. For example, if the parameters require the addition of an electrical system fault, fault manifestations will be set in the electrical system of the virtual aircraft, such as flashing dashboard indicators or partial equipment failure.

[0366] Electronic device 6 can adjust parameters based on the frequency of fault occurrence to control the rhythm of fault occurrence. If the parameters are set to a high fault frequency, various faults will occur more frequently during the virtual driving process, increasing the difficulty and challenge of training. Conversely, the number of faults will be reduced, reducing the complexity of the scenario.

[0367] Electronic device 6 can adjust parameters based on the environmental complexity to change the environmental settings in the scene. When the parameters indicate increasing environmental complexity, the weather may be adjusted to a harsh state, such as heavy rain or strong winds, and the terrain may be more complex, such as more obstacles or rugged terrain. If the parameters indicate decreasing environmental complexity, the weather may be adjusted to a better state, and the terrain may be flatter and more open.

[0368] By adjusting the fault types, fault frequencies, and environmental complexity of the backup virtual driving scenarios, a target virtual driving scenario is ultimately generated. This scenario closely matches the target user's current cognitive load level, providing a challenging yet low-inducing training environment, thereby enhancing the effectiveness of virtual driving training and the user's learning experience.

[0369] The flight training system provided in an embodiment of the present application includes electronic device 6, which is further configured to input physiological time-series data, EEG time-series data, eye movement image data, facial image data, and a target cognitive load level into a meta-learner; the meta-learner then obtains an initial cognitive load-scenario response model. The meta-learner then adjusts the initial cognitive load-scenario response model based on the physiological time-series data, EEG time-series data, eye movement image data, and facial image data to generate a target cognitive load-scenario response model corresponding to the target user, ensuring that the generated target cognitive load-scenario response model matches the target user. The target cognitive load level is input into the target cognitive load-scenario response model, and a scene complexity adjustment parameter is output. The accuracy of the output scene complexity adjustment parameter is ensured. These parameters are generated based on a precise analysis of the user's state and can effectively balance training difficulty and user tolerance. The complexity of the backup initial virtual driving scene is adjusted based on the scene complexity adjustment parameter to generate the target virtual driving scene. During the adjustment process, the system adjusts parameters based on the fault type, adding or replacing corresponding fault simulations in the scenario, such as engine failure or instrument malfunction. It also adjusts parameters based on the frequency of faults, controlling the intervals and frequency of faults to create a reasonable training rhythm. Furthermore, it adjusts parameters based on environmental complexity, adjusting environmental elements in the scenario, such as changing weather conditions or increasing the number of obstacles. Through these adjustments, the target virtual driving scenario closely matches the user's current cognitive load and training needs, providing the most suitable training environment. This prevents both frustration and fatigue caused by excessive training difficulty and ineffective training due to oversimplification, effectively improving training quality and efficiency and helping users better master flying skills.

[0370] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A flight training system, characterized in that: The flight training system includes a multi-axis gravity seat, a flight control stick, a button control panel, a parameter setting interface, a data acquisition device, a dynamic scene display device, and electronic equipment; wherein the multi-axis gravity seat, the flight control stick, the data acquisition device, and the dynamic scene display device are all communicatively connected to the electronic equipment; wherein: The flight joystick is used to receive operation action data input by the target user based on preset training instructions. The operation action data includes the execution order corresponding to each operation action and the operation force corresponding to each operation action; Button control panel, used to receive button action data input by the target user based on preset training instructions; An electronic device, configured to receive parameter setting data for setting flight parameters input by a target user based on preset training instructions; A multi-axis gravity seat, which is used to perform six-degree-of-freedom motion based on operation action data, button action data, and parameter setting data, thereby simulating the flight attitude of the aircraft; Data acquisition equipment, used to collect the target user's corresponding physiological time series data, EEG time series data, eye movement image data and facial image data during the training process; The electronic device inputs physiological time series data, EEG time series data, eye movement image data and facial image data into a preset workload recognition model, and outputs the current workload level corresponding to the target user; performs semantic recognition on the preset training instructions, and extracts key information from the preset training instructions; takes each key information as the initial node, the dependency relationship between each key information as the initial edge, and sets attribute information for each initial node to construct an initial graph structure; performs multiple convolution operations on the initial graph structure to obtain the initial node features corresponding to each initial node in the graph structure; generates an initial virtual driving scene based on the features of each initial node; abstracts the operation action data, button action data, and parameter setting data into update nodes, and establishes update edge connections with each initial node in the initial graph structure; sets corresponding attributes for each update edge; determines the weight and direction of each update edge according to the correlation between the operation action data, button action data, parameter setting data and the preset training instructions, and updates the attribute information of each updated node to generate a target graph structure; performs convolution operations on the target graph structure to obtain the target node features corresponding to each target node in the graph structure; and passes the target node feature matrix as input to the input layer of the preset recurrent neural network; For each time step, the input layer passes the target node feature matrix of the current time step to the hidden layer; the hidden layer calculates the hidden state of the current time step based on the current input and the hidden state of the previous time step, and outputs the current target state features; the current target state features are associated with each element in the initial virtual driving scene, and an association analysis is performed. Based on the analysis results, an adjustment decision is made for the initial virtual driving scene, and the initial virtual driving scene is adjusted to generate an alternative virtual driving scene; the complexity of the alternative initial virtual driving scene is adjusted according to the target cognitive load level to generate the target virtual driving scene.

2. The flight training system according to claim 1, characterized in that: The flight training system further includes an airflow control system, which includes a plurality of airflow valves and an airflow pressure sensor; the airflow control system is communicatively connected to the electronic device, wherein: The electronic device is configured to determine, based on the preset training instruction, external environmental data corresponding to the preset training instruction; and determine, based on the external environmental data and the flight attitude, a target airflow direction and a target airflow speed corresponding to the aircraft; The airflow control system is used to detect the current airflow direction and the current airflow speed based on each of the airflow pressure sensors; and control the opening or closing of each of the airflow valves according to the difference between the target airflow direction and the target airflow speed and the current airflow direction and the current airflow speed.

3. The flight training system according to claim 1, wherein: The multi-axis gravity seat includes a superconducting magnetic levitation device, a six-degree-of-freedom electric seat, and an adjustable seat belt array; wherein: The six-degree-of-freedom electric seat is mounted on the superconducting magnetic levitation device, and the six-degree-of-freedom electric seat is suspended above the superconducting magnetic levitation device through the Meissner effect generated by the superconducting material under a preset temperature environment; The six-degree-of-freedom electric seat is equipped with a linear drive unit consisting of multiple sets of permanent magnets and electromagnetic coils to achieve six-degree-of-freedom movement; The adjustable safety belt array is connected to the six-degree-of-freedom electric seat through multiple independent electric tightening devices, and the pressure applied to the target user is adjusted according to the flight posture.

4. The flight training system according to claim 1, wherein: The preset workload identification model includes a first feature identification network and a second feature identification network, and the electronic device is configured to: Inputting the physiological time series data and the EEG time series data into the first feature recognition network to generate time series modal features; Inputting the facial image data and the eye movement image data into the second feature recognition network to generate visual modality features; Fusing the temporal modal features and the visual modal features to generate target fused features; Based on the target fusion feature, the current workload level corresponding to the target user is output.

5. The flight training system according to claim 4, characterized in that: The electronic device is configured to use the visual modality features as a first query matrix, and the temporal modality features as a first key matrix and a first value matrix; Calculating a first dependency weight of the visual modality feature on the temporal modality feature; Using the temporal modality features as a second query matrix, and the visual modality features as a second key matrix and a second value matrix; calculating a second dependency weight of the temporal modality feature on the visual modality feature; Multiplying the time series modal feature by the first dependency weight to obtain a target time series feature; multiplying the visual modality feature by the second dependency weight to obtain a target visual feature; The target temporal features and the target visual features are combined to generate the target fusion features.

6. The flight training system according to claim 1, characterized in that: The key information includes at least one of flight mission type, flight area, and weather conditions.

7. The flight training system according to claim 1, characterized in that: The electronic device is further configured to input the physiological time series data, the EEG time series data, the eye movement image data, the facial image data, and the target cognitive load level into a meta-learner; The meta-learner obtains an initial cognitive load-scenario response model; the initial cognitive load-scenario response model includes an input layer, a hidden layer, and an output layer; the input layer is responsible for receiving data, and the number of neurons in the input layer is set according to the data feature dimension; the hidden layer is composed of multiple layers of neurons and processes the data through complex connection weights; The output layer outputs the final result; The meta-learner adjusts the initial cognitive load-scenario response model based on the physiological time series data, the EEG time series data, the eye movement image data, and the facial image data to generate a target cognitive load-scenario response model corresponding to the target user; The target cognitive load level is input into the target cognitive load-scenario response model, and a scenario complexity adjustment parameter is output; the scenario complexity adjustment parameter includes fault type adjustment, fault frequency adjustment, and environment complexity adjustment: The complexity of the backup initial virtual driving scene is adjusted based on the scene complexity adjustment parameter to generate a target virtual driving scene.

Images