Aircraft cockpit simulation method and equipment for low-altitude typical scene

Abstract

The invention provides a low-altitude typical scene-oriented aircraft cockpit simulation method, system and device and a storage medium, and the method comprises the steps: constructing a dynamic extensible low-altitude scene library through fusing mixed reality (MR) and a high-precision digital twin model; in combination with a man-machine in-the-loop multimode interaction technology (eye movement-gesture-voice cooperation), the operation naturalness is improved; and an intelligent optimization algorithm driven by reinforcement learning is introduced, adaptive adjustment of flight control parameters is realized, and finally a'perception-decision-evaluation 'closed-loop training system is formed. According to the invention, digital twinning, mixed reality (MR), multi-modal sensing and adaptive control technologies are fused, pilot low-altitude flight skill training, man-machine work efficiency evaluation and intelligent aircraft control system verification are carried out, and the method can be expanded to UAM operator training and unmanned aircraft simulation testing. The technical barrier of low-altitude flight skill training at present is solved, and the technical blank of low-altitude dynamic scene simulation is filled up.

Description

Technical Field

[0001] The present invention relates to the technical field of flight simulation, and specifically, to a flight simulator cockpit simulation method, system, device and storage medium for low-altitude typical scenarios. Background Art

[0002] With the rapid development of the low-altitude economy (such as drone logistics, urban air traffic, emergency rescue, etc.), pilots' situation awareness and emergency response capabilities in complex low-altitude scenarios (such as densely built-up areas, mountain valleys, and adverse weather) face higher requirements. Traditional flight simulation systems are mostly based on fixed scenarios and preset scripts, and it is difficult to dynamically reconstruct elements such as the distribution of low-altitude obstacles and real-time meteorological disturbances, resulting in a significant difference between the training environment and the actual working conditions. In the prior art, although digital twin technology has been applied to flight simulation, there are bottlenecks in aspects such as real-time virtual-real interaction (delay > 50ms), multi-modal human-machine collaboration (gesture recognition rate < 80%), and adaptive control (static parameters cannot adapt to individual differences). In addition, the unique sudden risks of low-altitude flight (such as bird flock collisions and avoidance of micro drones) pose higher challenges to the environmental perception accuracy (existing lidar error ≥ 5cm) and decision-making response speed (obstacle avoidance delay ≥ 1s) of the simulation system. Summary of the Invention

[0003] The purpose of the present invention is to provide a flight simulator cockpit simulation method, system, device and storage medium for low-altitude typical scenarios, aiming to solve the current technical barriers in low-altitude flight skills training, integrating digital twin, mixed reality (MR), multi-modal sensing and adaptive control technologies, conducting pilots' low-altitude flight skills training, human-machine ergonomics evaluation and intelligent aircraft control system verification, and being able to be extended to the training of urban air traffic (UAM) operators and the simulation test of unmanned aerial vehicles.

[0004] The present invention discloses a flight simulator cockpit simulation method for low-altitude typical scenarios, including:

[0005] Collecting multi-source data of low-altitude scenarios and constructing a digital twin mixed reality flight simulation model;

[0006] Establishing a human-in-the-loop multi-modal interaction between the flight simulation model and the pilot;

[0007] Obtaining the human-machine interaction data generated during the human-in-the-loop multi-modal interaction, inputting it into the intelligent optimization control model of the aircraft, and performing intelligent optimization control of the aircraft through the output result of the model.

[0008] Among them, collecting multi-source data of low-altitude scenarios and constructing a digital twin mixed reality flight simulation model includes:

[0009] Collect multi-source data of low-altitude scenarios, and establish an aircraft dynamics simulator and customized cockpits based on different aircraft models;

[0010] Install a multi-modal human-computer interaction sensor array on the aircraft dynamics simulator;

[0011] Provide flight assistance and flight parameter optimization for the aircraft dynamics simulator;

[0012] Construct a virtual-real fusion visualization display interface for the aircraft dynamics simulator.

[0013] Among them, collecting multi-source data of low-altitude scenarios, establishing an aircraft dynamics simulator and customized cockpits based on different aircraft models includes:

[0014] Collect multi-source data of low-altitude scenarios, establish an aircraft dynamics simulator and customized cockpits based on different aircraft models, generate urban scenarios based on the BIM and GIS fusion technology as the aircraft simulation model for digital twin mixed reality;

[0015] Define the mapping relationship between pilot operation instructions and flight control logic;

[0016] Adopt the Unreal Engine 5 Nanite virtualized micro-polygon geometry technology to perform dynamic global illumination (Lumen system) and real-time fluid simulation (Chaos physics engine) on the aircraft simulation model to complete the rendering of the virtual simulation engine;

[0017] Construct a spatio-temporal multi-modal database for storing historical flight trajectories, sensor calibration parameters, and environmental disturbance patterns.

[0018] Among them, installing a multi-modal human-computer interaction sensor array on the aircraft dynamics simulator includes:

[0019] Collect the gyro zero bias value and the pilot's finger bending degree of the aircraft dynamics simulator through a state perception device;

[0020] Collect the hydraulic actuator pressure value of the aircraft dynamics simulator through the CAN bus;

[0021] Record the hot spots of the pilot's attention distribution through an augmented reality headset device with an embedded eye tracking unit.

[0022] Among them, providing flight assistance and flight parameter optimization for the aircraft dynamics simulator includes:

[0023] Realize real-time tracking of the pilot's head and hands through machine vision coupled with UWB ultra-wideband positioning technology (accuracy ±10 cm);

[0024] Dynamically correct the spatial coordinate offset of the MR scene by combining the AI vision behavior feature extraction algorithm;

[0025] Based on the simulation results of the digital twin model, adopt the reinforcement learning algorithm to generate an optimized flight parameter solution.

[0026] Among them, establish the man-in-the-loop multi-modal interaction between the aircraft simulation model and the pilot, including:

[0027] Multi-modal input, including gesture recognition, eye movement control, and voice interaction;

[0028] Haptic feedback, including vibration feedback and force feedback;

[0029] Multi-source data fusion, adopt the timestamp alignment algorithm to synchronize sensor data, and weight and fuse the priorities of gesture, eye movement, and voice commands through the attention mechanism.

[0030] Among them, obtain the human-machine interaction data generated during the man-in-the-loop multi-modal interaction, input it into the aircraft intelligent optimization control model, and perform intelligent optimization control of the aircraft through the model output results, including:

[0031] Construct a new network architecture based on LSTM + Kalman filter to predict the pilot's operation intention, and automatically compensate the rudder surface deflection when continuous heading deviation is detected;

[0032] Develop a digital twin comparison engine, calculate the Hausdorff distance between the actual and virtual flight trajectories, and trigger haptic alarms when the limit is exceeded;

[0033] Preset failure modes and design failure recovery strategies;

[0034] Dynamically adjust the training difficulty based on the Q-learning algorithm, and adopt the model pruning technology to compress the number of neural network parameters.

[0035] The present invention discloses an aircraft cockpit simulation system for low-altitude typical scenarios, including:

[0036] A digital twin mixed reality module for collecting multi-source data of low-altitude scenarios and constructing a digital twin mixed reality aircraft simulation model;

[0037] A man-in-the-loop multi-modal interaction module for establishing the man-in-the-loop multi-modal interaction between the aircraft simulation model and the pilot;

[0038] A flight intelligent optimization control module for obtaining the human-machine interaction data generated during the man-in-the-loop multi-modal interaction, inputting it into the aircraft intelligent optimization control model, and performing intelligent optimization control of the aircraft through the model output results.

[0039] The present invention discloses a computer device, including an input / output unit, a memory, and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor executes the steps in the method of the foregoing embodiment.

[0040] The present invention discloses a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps in the method of the foregoing embodiment.

[0041] Different from the prior art, a flight simulator method for a flight cockpit in typical low-altitude scenarios of the present invention constructs a dynamically scalable low-altitude scenario library by integrating mixed reality (MR) and a high-precision digital twin model; combines a human-in-the-loop multi-modal interaction technology (eye movement-gesture-voice collaboration) to improve the naturalness of operations; introduces an intelligent optimization algorithm driven by reinforcement learning to achieve adaptive adjustment of flight control parameters, and finally forms a "perception-decision-evaluation" closed-loop training system. The present invention integrates digital twin, mixed reality (MR), multi-modal sensing, and adaptive control technologies for pilot low-altitude flight skill training, human-machine ergonomics evaluation, and intelligent aircraft control system verification, and can be extended to urban air mobility (UAM) operator training and unmanned aircraft simulation testing, solving the technical barriers faced in current low-altitude flight skill training and filling the technical gap in low-altitude dynamic scenario simulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The present invention will be further described below in conjunction with the drawings and embodiments. In the drawings:

[0043] Figure 1 is a schematic flow chart of a flight simulator method for a flight cockpit in typical low-altitude scenarios of the present invention.

[0044] Figure 2 is a schematic structural diagram of a flight simulator system for a flight cockpit in typical low-altitude scenarios of the present invention.

[0045] Figure 3 is a schematic structural diagram of a non-transitory computer-readable storage medium storing computer instructions provided by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] In order to have a clearer understanding of the technical features, objectives, and effects of the present invention, the specific embodiments of the present invention will be described in detail below with reference to the drawings.

[0047] Please refer to Figure 1 , the present invention discloses a flight simulator method for a flight cockpit in typical low-altitude scenarios, including:

[0048] S110: Collect multi-source data of low-altitude scenarios and construct a flight vehicle simulation model for digital twin mixed reality.

[0049] Step S110 specifically includes:

[0050] S111: Collect multi-source data of low-altitude scenarios, and establish a flight vehicle dynamics simulator and a customized cockpit based on different flight vehicle models.

[0051] In the present invention, the multi-source data of low-altitude multi-scenarios at least includes 4D information of airspace, satellite remote sensing images and meteorological dynamic data. The constructed flight vehicle dynamics simulator is a 7-degree-of-freedom multi-servo hydraulic mechanism.

[0052] Based on the BIM and GIS fusion technology to generate urban scenes, the present invention adopts a LOD generation method of a multi-layer pyramid adaptive tree structure that couples to generate texture maps with high integrity and high occupancy rate, specifically including:

[0053] Let the integrity of the texture map be C and the occupancy rate be U. The integrity C can be measured by the integrity of the texture blocks and the tightness of the arrangement, and the occupancy rate U is calculated by the ratio of the area of the texture blocks to the total texture area:

[0054]

[0055] Among them, T i represents the i-th texture map, and n is the total number of texture maps.

[0056] Let the multi-layer pyramid have L layers, and the resolution of each layer decreases in turn. The resolution of the l-th layer can be expressed as:

[0057]

[0058] Among them, Rmax represents the highest resolution, l is the layer index, and 0 ≤ l < L.

[0059] Let the viewing distance be D and the importance of the model be I. According to the viewing distance and importance, select an appropriate level of detail l for rendering:

[0060]

[0061] Among them, Dmin is the minimum viewing distance, represents rounding down.

[0062] Define the mapping relationship between the pilot's operation instructions and the flight control logic, and its formula is as follows:

[0063]

[0064] Among them, the input variables include:

[0065] The pilot's operation command, denoted as u = [δ elevator , δ throttle , δ rudder T ;

[0066] The current flight state of the aircraft, denoted as x = [V, h, θ, φ, ψ] T ;

[0067] The output variables include:

[0068] The position of the control surface of the aircraft: denoted as y control = [δ elevator_out , δ aileron , δ rudder_out T ;

[0069] The thrust setting of the engine: T;

[0070] K elevator : The gain coefficient of the joystick position.

[0071] K speed : The gain coefficient of the speed feedback.

[0072] K height : The gain coefficient of the altitude feedback.

[0073] K aileron : The gain coefficient of the aileron input.

[0074] K roll : The gain coefficient of the roll angle feedback.

[0075] K rudder : The gain coefficient of the pedal position.

[0076] K yaw : The gain coefficient of the yaw angle feedback.

[0077] K throttle : The gain coefficient of the throttle position.

[0078] In addition, the pilot's operation commands include:

[0079] The joystick position (pitch angle command): δ elevator ;

[0080] The throttle position (thrust command): δ throttle ;

[0081] The pedal position (yaw command): δ rudder ;

[0082] The current flight state of the aircraft: ​​

[0083] Speed: V;

[0084] Height: h;

[0085] Attitude (pitch angle, roll angle, yaw angle): θ, φ, ψ.

[0086] Furthermore, the present invention adopts the Unreal Engine 5 Nanite virtualized micro-polygon geometry technology to achieve dynamic global illumination (Lumen system) and real-time fluid simulation (Chaos physics engine). Its innovation lies in encapsulating a special aircraft cabin simulation engine on this basis and optimizing the accuracy of the fluid model using aircraft dynamics.

[0087] Meanwhile, a spatio-temporal multi-modal database is constructed to store historical flight trajectories, sensor calibration parameters, and environmental disturbance patterns.

[0088] S112: Install a multi-modal human-machine interaction sensor array for the aircraft dynamics simulator.

[0089] Specifically, it includes:

[0090] Collect the gyro zero bias value and the pilot's finger bending degree of the aircraft dynamics simulator through a state perception device.

[0091] Among them, the state perception device includes a nine-axis MEMS inertial unit and a flexible strain sensor. The nine-axis MEMS inertial unit is used to control the gyro zero bias stability ≤ 0.5° / h, and the flexible strain sensor is used to control the finger bending degree detection accuracy to reach ±1°.

[0092] Collect the hydraulic actuator pressure value of the aircraft dynamics simulator through the CAN bus. The hydraulic actuator pressure value range is 0 - 20 MPa, and the sampling rate is 1 kHz;

[0093] Record the pilot's attention distribution hotspots through an augmented reality headset device with an embedded eye tracking unit.

[0094] S113: Provide flight assistance and flight parameter optimization for the aircraft dynamics simulator.

[0095] Specifically, real-time tracking of the pilot's head and hands is achieved through machine vision coupled with UWB ultra-wideband positioning technology (accuracy ±10 cm), and the spatial coordinate offset of the MR scene is dynamically corrected by combining the AI vision behavior feature extraction algorithm; based on the simulation results of the digital twin model, a flight parameter optimization scheme (such as the angle of attack adjustment amount Δα = current value × 0.2) is generated using the reinforcement learning algorithm (PPO strategy) and automatically updated to the control system.

[0096] S114: Construct a virtual-real fusion visualization display interface for the aircraft dynamics simulator.

[0097] Specifically include:

[0098] Construct a three-dimensional thermal map of the experimental process to display the pilot's operation pressure distribution with color gradients, where the high-frequency accidental touch areas can be represented by red.

[0099] Set up a real-time performance monitoring panel to display the dynamic display frame rate (≥90fps), end-to-end latency (≤20ms), and GPU video memory occupancy rate (threshold ≤ 80%).

[0100] Set up an MR-assisted annotation system to overlay virtual waypoints and no-fly zone boundaries through an augmented reality headset device. The virtual waypoints and no-fly zone boundaries can be warned by setting translucent red cubes.

[0101] S120: Establish a man-in-the-loop multi-modal interaction between the aircraft simulation model and the pilot.

[0102] The man-in-the-loop multi-modal interaction includes:

[0103] Multi-modal input channels:

[0104] Data glove: Integrated with 16 bending sensors and 9-axis IMU, supporting 27 standard gesture recognitions, such as three-finger pinch mapping to throttle fine-tuning, with recognition latency ≤ 30ms.

[0105] Eye movement control: Define that the fixation point lasts for 1s to trigger virtual button selection, and when the pupil diameter change rate > 15%, it is determined as a stress state.

[0106] Voice interaction: Adopt an end-to-end speech recognition model (Conformer architecture), supporting the parsing of mixed Chinese and English instructions, with an accuracy rate ≥ 85% when the noise ≥ 70dB.

[0107] Haptic feedback mechanism:

[0108] Vibration feedback: Provide hierarchical prompts according to the flight state. The specific levels include: Mild turbulence: 50Hz intermittent vibration; Severe bump: 120Hz continuous vibration.

[0109] Force feedback: Simulate the resistance of the joystick through a tendon-driven device. Among them, the maximum output force is 20N, the resolution is 0.1N, and the resistance suddenly increases by 300% during stall warning.

[0110] Multi-source data fusion: Use the timestamp alignment algorithm (PTP precision clock protocol) to synchronize sensor data, and fuse the priorities of gestures, eye movements, and voice commands through an attention mechanism. The weight ratio is set to 3:2:1.

[0111] S130: Obtain the human-machine interaction data generated during the human-in-the-loop multi-modal interaction process, input it into the intelligent optimization control model of the aircraft, and perform intelligent optimization control of the aircraft through the output results of the model.

[0112] Construct an adaptive control algorithm based on the data-driven of LSTM neural network. Specifically, it includes:

[0113] Construct a new network architecture based on LSTM + Kalman filter to predict the pilot's operation intention. When a continuous course deviation is detected, such as >5° for more than 3s, automatically compensate the rudder surface deflection amount. For example, the compensation rate linearly decreases from 100% to 30% with the training duration.

[0114] Develop a digital twin comparison engine, calculate the Hausdorff distance between the actual and virtual flight trajectories, set the threshold to ≤2m, and trigger a tactile alarm when the limit is exceeded.

[0115] Preset failure modes and design a failure recovery strategy.

[0116] Specifically, 12 failure modes including engine thrust loss and pitot tube icing are preset, and the fault injection delay is less than or equal to 100ms.

[0117] The failure recovery strategy is designed as a two-layer recovery strategy. The primary is automatic modification, such as restarting the avionics system. After failure, start the backup navigation link.

[0118] Based on the Q-learning algorithm, dynamically adjust the training difficulty. The obstacle density increases with the increase of the success rate, and the increase formula: D _new = D _old ×(1 + √(success rate / 100).

[0119] Adopt model pruning technology to compress the number of neural network parameters, reduce the GPU power consumption by 45%, and achieve energy consumption optimization.

[0120] In addition, the present invention also includes the steps of constructing multi-screen collaborative display, specifically including:

[0121] The main screen (55-inch 4K OLED) displays the synthetic vision (SVG), which fuses the radar point cloud and the camera image.

[0122] The side screens (2 pieces of 27-inch LCD) display the virtual instrument (imitating the Garmin G5000 layout) and the task progress dashboard in split screens.

[0123] Refresh rate synchronization technology: Eliminate screen tearing through G-Sync, ensure the rendering priority of key frames, and set the update period of the course scale to ≤10ms.

[0124] Set up a light environment simulation unit, including:

[0125] An adjustable spectrum LED array is adopted, with an adjustable color temperature of 2500 - 6500K, simulating the illumination change of day and night alternation, and an illuminance gradient of 0.1 - 1000 lux.

[0126] Strong light suppression: When the HMD detects direct sunlight, it automatically reduces the screen brightness and increases the contrast.

[0127] Meanwhile, in the solution of the present invention, a data security and traceability function is set, including:

[0128] Operation record blockchain evidence storage: Generate Merkle tree hash values every 10 seconds, and achieve audit traceability through smart contracts;

[0129] Sensitive data desensitization processing: Adopt homomorphic encryption technology to protect physiological index data, such as heart rate and pupil diameter, with a decryption delay ≤ 5 ms.

[0130] In other embodiments of the present invention, an interface dynamic optimization function is also set, including:

[0131] Automatically adjust the control layout based on the eye movement heat map, and shift the high-frequency operation buttons towards the gaze focus area, where the set maximum offset distance ≤ 15% of the screen width;

[0132] Color self-adaptation: Detect the user's color weakness characteristics through calibration tests, and automatically switch to a high-contrast color matching scheme, such as replacing red and green with blue and yellow.

[0133] As Figure 2 As shown, the present invention proposes a flight simulator system 200 for low-altitude typical scenarios, including:

[0134] A digital twin mixed reality module 210, which is used to collect multi-source data of low-altitude scenarios and construct a digital twin mixed reality flight vehicle simulation model;

[0135] A human-in-the-loop multi-modal interaction module 220, which is used to establish a human-in-the-loop multi-modal interaction between the flight vehicle simulation model and the pilot;

[0136] A flight intelligent optimization control module 230, which is used to obtain the human-machine interaction data generated during the human-in-the-loop multi-modal interaction process, input it into the flight vehicle intelligent optimization control model, and perform intelligent optimization control of the flight vehicle through the output results of the model.

[0137] To implement the embodiments, the present invention also proposes an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps in the methods of the foregoing technical solutions.

[0138] AsFigure 3 As shown, the non-transitory computer-readable storage medium includes a memory 810 for instructions, an interface 830, and the instructions can be executed by a processor 820 to complete a method. Optionally, the storage medium may be a non-transitory computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0139] To implement the embodiments, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method as in the embodiments of the present invention is implemented.

[0140] In the description of this specification, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, without contradiction, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples.

[0141] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise specifically and clearly defined.

[0142] Any process or method description in the flowchart or described in other ways herein may be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a customized logic function or process. The scope of the preferred embodiments of the present invention includes additional implementations, where the functions may be executed in a substantially simultaneous manner or in a reverse order according to the involved functions, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0143] The logic and / or steps represented in the flowchart or otherwise described herein can, for example, be considered a definitional sequence of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection unit (electronic device) having one or more wirings, a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable medium on which the program can be printed, as the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpretation, or other suitable processing as necessary, and then storing it in a computer memory.

[0144] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the described embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.

[0145] Those of ordinary skill in the art of the present technology can understand that all or part of the steps carried by the method of the described embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0146] In addition, in each embodiment of the present invention, each functional unit can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0147] The storage medium mentioned above can be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the embodiments within the scope of the present invention.

Claims

1. A flight simulator method for low-altitude typical scenarios of an aircraft, characterized in that Including: Collect multi-source data of low-altitude scenarios and build an aircraft simulation model for digital twin mixed reality; Establish a man-in-the-loop multi-modal interaction between the aircraft simulation model and the pilot; Obtain the human-machine interaction data generated during the man-in-the-loop multi-modal interaction, input it into the aircraft intelligent optimization control model, and perform intelligent optimization control of the aircraft through the model output results.

2. The aircraft cockpit simulation method for low-altitude typical scenarios according to claim 1, characterized in that, Collect multi-source data of low-altitude scenarios and build an aircraft simulation model for digital twin mixed reality, including: Collect the multi-source data of the low-altitude scenarios, establish an aircraft dynamics simulator and a customized cockpit based on different aircraft models; Install a multi-modal human-machine interaction sensor array on the aircraft dynamics simulator; Provide flight assistance and flight parameter optimization for the aircraft dynamics simulator; Build a virtual-real fusion visualization display interface for the aircraft dynamics simulator.

3. The method for simulating an aircraft cockpit for low-altitude typical scenarios according to claim 2, characterized in that, Collect the multi-source data of the low-altitude scenarios, establish an aircraft dynamics simulator and a customized cockpit based on different aircraft models, including: Collect the multi-source data of the low-altitude scenarios, establish an aircraft dynamics simulator and a customized cockpit based on different aircraft models, generate an urban scene based on the BIM and GIS fusion technology, and use it as the aircraft simulation model for digital twin mixed reality; Define the mapping relationship between pilot operation instructions and flight control logic; Use the Unreal Engine 5 Nanite virtualized micro-polygon geometry technology to perform dynamic global illumination (Lumen system) and real-time fluid simulation (Chaos physics engine) on the aircraft simulation model to complete the rendering of the virtual simulation engine. Build a spatio-temporal multi-modal database for storing historical flight trajectories, sensor calibration parameters, and environmental disturbance patterns.

4. The method for simulating an aircraft cockpit for low-altitude typical scenarios according to claim 2, wherein, Install a multi-modal human-machine interaction sensor array on the aircraft dynamics simulator, including: Collect the gyro zero bias value of the aircraft dynamics simulator and the pilot's finger bending degree through a state perception device; Collect the hydraulic actuator pressure value of the aircraft dynamics simulator through the CAN bus; Record the hot spots of the pilot's attention distribution through an augmented reality headset device with an embedded eye movement tracking unit.

5. The method for simulating an aircraft cockpit for typical low-altitude scenarios according to claim 2, wherein, Provide flight assistance and flight parameter optimization for the aircraft dynamics simulator, including: Realize real-time tracking of the pilot's head and hands through machine vision coupled with UWB ultra-wideband positioning technology (accuracy ±10 cm); Dynamically correct the spatial coordinate offset of the MR scene by combining the AI vision behavior feature extraction algorithm; Generate a flight parameter optimization scheme using the reinforcement learning algorithm based on the simulation results of the digital twin model.

6. The method for simulating an aircraft cockpit for low-altitude typical scenarios according to claim 1, wherein Establish a man-in-the-loop multi-modal interaction between the aircraft simulation model and the pilot, including: Multi-modal input, including gesture recognition, eye movement control, and voice interaction; Haptic feedback, including vibration feedback and force feedback; Multi-source data fusion, use the timestamp alignment algorithm to synchronize sensor data, and weight and fuse the priorities of gestures, eye movements, and voice commands through the attention mechanism.

7. The method for simulating an aircraft cockpit for low-altitude typical scenarios according to claim 1, wherein Obtain the human-machine interaction data generated during the human-in-the-loop multi-modal interaction, input it into the intelligent optimization control model of the aircraft, and perform intelligent optimization control of the aircraft through the model output results, including: Construct a new network architecture based on LSTM + Kalman filter to predict the pilot's operation intention, and automatically compensate the rudder surface deflection when continuous course deviation is detected; Develop a digital twin comparison engine to calculate the Hausdorff distance between the actual and virtual flight trajectories, and trigger a tactile alarm when the limit is exceeded; Preset failure modes and design failure recovery strategies; Dynamically adjust the training difficulty based on the Q-learning algorithm, and use model pruning technology to compress the number of neural network parameters.

8. An aircraft cockpit simulation system for typical low-altitude scenarios, characterized in that, Including: A digital twin mixed reality module for collecting multi-source data of low-altitude scenes and constructing a digital twin mixed reality aircraft simulation model; A human-in-the-loop multi-modal interaction module for establishing a human-in-the-loop multi-modal interaction between the aircraft simulation model and the pilot; A flight intelligent optimization control module for obtaining the human-machine interaction data generated during the human-in-the-loop multi-modal interaction, inputting it into the intelligent optimization control model of the aircraft, and performing intelligent optimization control of the aircraft through the model output results.

9. A computer device, characterized in that, Including an input-output unit, a memory, and a processor. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by the processor, the processor executes the steps in any one of the methods described in claims 1 to 7.

10. A storage medium storing computer-readable instructions, characterized in that, When the computer-readable instructions are executed by one or more processors, one or more processors execute the steps in any one of the methods described in claims 1 to 7.

Images