An aircraft cockpit simulation system and a failure simulation training method thereof

Abstract

The present application relates to a kind of aircraft cockpit simulation system and its failure simulation training method, the system includes simulation cockpit, for simulating cockpit and making student on it to carry out simulation operation;Teaching terminal, instructor operates on it to issue setting training content;Simulation server end, for providing the model of various components of aircraft;Serial bus, connecting teaching terminal, simulation server end, and by an interface manager connects simulation cockpit;Serial bus is used for carrying out the data interaction between teaching terminal, simulation server end and simulation cockpit;Simulation server end will be simulated by simulation and form simulation video signal through interface manager and send to simulation cockpit, simulation server end will be simulated by simulation and form simulation audio signal and send to audio equipment, audio equipment is used with simulation cockpit. According to the scheme of the present application, the problem of unstable signal transmission process pressure and simulation training display effect in current simulation system is solved.

Description

Technical Field

[0001] This invention generally relates to the field of computer simulation technology. More specifically, this invention relates to an aircraft cockpit simulation system and a fault simulation training method thereof. Background Technology

[0002] Flight simulators are devices used to train pilots, typically consisting of a cockpit, interface equipment, various instruments, a visual system, and a training computer. The computer serves as the control center of the flight simulator. During training, trainees can perform various operations from the cockpit: turning on the power switch, pushing and pulling the throttle, operating the flight stick and rudder, and obtaining various data such as flight speed, distance traveled, position, altitude, wind direction, and wind speed. The visual system provides a simulated environment, allowing trainees to experience what it's like to be in an airplane, feeling the sensations of diving, pitching, and circling, and seeing various objects above and below the aircraft (clouds, fog, rivers, buildings). Various flight environments can also be set to comprehensively train skills and learn and master challenging and dangerous maneuvers.

[0003] Flight simulators possess significant advantages such as energy efficiency, economy, safety, lack of limitations due to location and weather conditions, shortened training cycles, reduced training costs, and improved training efficiency, playing a crucial role in pilot training. Currently, various flight simulators exist in China, falling into two main categories: imported and domestically developed. Imported civilian aircraft flight simulators primarily include Boeing and Airbus series. Domestically developed flight simulators are relatively few, and representative ones are mainly used for basic pilot training.

[0004] Current domestic flight simulation training devices not only require video to simulate what trainees see from various perspectives during flight, but also supplement it with audio signals to allow trainees to perceive the actual operating environment in real time. This results in significant bus resource consumption for both video and audio signals. A Chinese utility model patent with authorization announcement number CN218525011U, entitled "A Modular Aircraft Flight Control Development and Simulation Integrated System," discloses the composition of a flight simulator. This system includes a visual simulation computer, a simulation management computer, a cockpit simulator, a real-time aircraft model and onboard equipment simulation computer, a flight control computer simulator, and an integrated computer, all interconnected via fiber optic network switches and reflective memory cards for real-time communication. However, in this system, the simulation and data transmission of visual conditions are uniformly completed by the visual simulation computer, and the simulation and transmission of video and audio signals require substantial bus resources.

[0005] Therefore, in order to ensure the real-time performance and synchronization of signals in this aircraft flight simulation training device, the data transmission bus faces a large data transmission pressure, which may lead to a decrease in signal stability and seriously affect the effect of flight simulation training.

[0006] Therefore, the main problems with current aircraft flight simulation training devices are high pressure during signal transmission and unstable simulation training display effects. Summary of the Invention

[0007] To address one or more of the aforementioned technical problems, this invention proposes separating audio and video signals via separate wiring. Signals requiring high synchronization are transmitted via a bus, while audio signals can be directly sent to the audio device through a simulation server. This approach not only reduces the pressure on bus data transmission but also makes the pilot's simulated flight experience more realistic and closer to the actual situation. Therefore, this invention provides solutions in the following aspects.

[0008] In a first aspect, the present invention provides an aircraft simulation system, comprising: a simulated cockpit for simulating a cockpit and allowing trainees to perform simulated operations thereon; a teaching terminal for instructors to operate on to publish and set training content; a simulation server for providing models of various aircraft components; a serial bus connecting the teaching terminal and the simulation server, and connected to the simulated cockpit via an interface manager; the serial bus is used for data interaction between the teaching terminal, the simulation server, and the simulated cockpit; wherein the simulation server transmits simulated video signals generated during simulation to the simulated cockpit via the interface manager, and transmits simulated audio signals generated during simulation to an audio device, wherein the audio device is used in conjunction with the simulated cockpit.

[0009] In one embodiment, the simulated cockpit communicates with the interface manager via video signals, Ethernet signals, and hardwired signals.

[0010] In one embodiment, the serial bus is a 1394 bus.

[0011] In one embodiment, the simulated cockpit is equipped with a simulated cockpit subsystem, the interface manager is equipped with an interface management subsystem, the simulation server is equipped with a software integration and environment operation subsystem, a sound simulation subsystem, and an aircraft simulation subsystem, and the teaching terminal is equipped with a teaching control subsystem. The software integration and environment operation subsystem is used to schedule the various subsystems, control data interaction between them, and control communication between the interface management subsystem and the aircraft simulation subsystem. The simulated cockpit subsystem supports the simulated cockpit hardware, converts student operations into hardwired or Ethernet signals for transmission to the interface management subsystem, and receives video signals from the interface management subsystem for image display. The aircraft simulation subsystem runs simulation models of at least one aircraft component. The sound simulation subsystem simulates at least one sound during flight. The teaching control subsystem sets training content. The interface management subsystem performs data conversion and transmission.

[0012] In one embodiment, the teaching control subsystem is used to set training content, including: the teaching control subsystem is used to set initial parameters for simulation models of at least one component of the aircraft.

[0013] In one embodiment, the aircraft simulation subsystem is used to run simulation models of at least one component of the aircraft, including: receiving control signals from the simulated cockpit subsystem and initial parameters of the model from the teaching and control subsystem, running the corresponding setup simulation model, and outputting data to other subsystems, wherein the control signals are generated by the student through operating the simulated cockpit.

[0014] In one embodiment, the aircraft simulation subsystem includes integrated processing software and a DDS soft bus; the integrated processing software interacts with component simulation models, algorithm models, and flight dynamics models via the DDS soft bus.

[0015] In one embodiment, the component simulation model is connected to the DDS soft bus via AMESim and Simulink interfaces, the algorithm model is connected to the DDS soft bus via a C / C++ interface, and the flight dynamics model is connected to the DDS soft bus via a FlightSim interface.

[0016] In one embodiment, the integrated processing software includes flight management software, central alarm software, and central maintenance software.

[0017] In one embodiment, the aircraft simulation subsystem is further configured to run an atmospheric data model, which provides flight parameters calculated based on atmospheric data.

[0018] In one embodiment, the atmospheric data model is used to receive the corresponding parameters set by the teaching control subsystem and the sensor information calculated by the aircraft simulation subsystem, and then calculate the flight control parameters, which are further calculated by the aircraft simulation subsystem.

[0019] In one embodiment, the simulated cockpit subsystem includes a top control panel, a front control panel, a central control panel, a left control panel, a right control panel, and a control mechanism within the cockpit, wherein the control mechanism includes a control stick, a control wheel, pedals, and a throttle lever.

[0020] In one embodiment, the interface management subsystem is used to complete the signal acquisition, excitation, and data conversion and transmission of the top control panel, front control panel, central control panel, left control panel, and right control panel of the cockpit in the simulated cockpit subsystem; the interface management subsystem is also used to realize the conditioning, adaptation, and transmission of electrical signals of the control mechanism in the simulated cockpit subsystem.

[0021] In one embodiment, the component simulation model includes one or more of the following: hydraulic system model, environmental control system model, fuel system model, power unit model, auxiliary power unit model, oxygen system model, environmental protection system model, control model, and lighting system model; the control model includes a landing gear control system model and a door control system model.

[0022] In one embodiment, the top control panel includes one or more of the following control panels: emergency positioning control panel, cabin lighting control panel, avionics start control panel, flight control system control panel, hydraulic system control panel, backup shutdown control panel, power system control panel, engine start control panel, fuel system control panel, external lighting control panel, windshield wiper control switch, landing lighting control panel, fire protection system control panel, cabin voice monitoring control panel, electromechanical management system control panel, lifesaving system control panel, oxygen system control panel, anti-icing system control panel, gas supply system control panel, air conditioning system control panel, and pressure regulation system control panel.

[0023] In one embodiment, the forward control panel includes one or more of the following: a left warning light, a right warning light, a left display control panel, a right display control panel, an autoflight control panel, an approach warning light, a landing gear control handle, a landing gear light, and an autobrake selector panel.

[0024] In one embodiment, the central control console includes one or more of the following: emergency stop brake, horizontal stabilizer position indicator, multifunction display, horizontal stabilizer trim control handle, speed brake handle, horizontal stabilizer trim cut-off switch, flap handle, left and right trackballs, left and right multifunction keyboards, normal stop switch, flap and slat overrun control panel, radio tuning unit, fire suppression control panel, throttle control console, audio control panel, trim control panel, selection switch panel, cabin door control panel, cockpit airdrop control panel, and electronic warfare control panel.

[0025] In one embodiment, the left control panel includes a driver's side oxygen mask control panel, a left dimming control panel, a driver's side front wheel steering handle, a left head-up display control panel, a key control box, a time key control box, and a headphone / phone jack assembly.

[0026] In one embodiment, the right control panel includes a passenger-side oxygen mask control panel, a right dimming control panel, a passenger-side front wheel steering handle, a right head-up display control panel, an oxygen shut-off valve switch, and a task loading / unloading card.

[0027] In one embodiment, each control board, panel, and console in the simulated cockpit subsystem is implemented through physical simulation components and / or virtual simulation interfaces.

[0028] In one embodiment, the aircraft simulation subsystem further includes: an electrical system model for data interaction with a hydraulic system model, a fuel system model, a power unit model, an oxygen system model, a landing gear system model, a door system model, and a lighting system model; the electrical system model is also used for data interaction with an avionics system model, a flight control system model, an engine model, and an electromechanical management system model; and the electrical system model is also used for state interconnection with electrical control panels.

[0029] In one embodiment, the aircraft simulation subsystem further includes a fuel system model, which is used to interact with the engine model, auxiliary power unit model, electromechanical management system model, power supply system model, teaching control subsystem, and fuel control panel.

[0030] In one embodiment, the aircraft simulation subsystem further includes a landing gear retraction system model, which is used to interact with the landing gear control handle, hydraulic system model, power supply system model, and electromechanical management system.

[0031] In one embodiment, the interface manager includes: an image generation computer connected to the front control panel and the central console via a DVI signal; and an interface computer for connecting to the front control panel, the central console, the top control panel, the left console, and the right console via a network switch.

[0032] In one embodiment, the interface manager further includes a power control box, which is connected to the front control panel, the central control panel, the top control panel, the left control panel, and the right control panel respectively, for providing DC power supply.

[0033] In one embodiment, the sound simulation subsystem is used to simulate at least one sound during flight, including: environmental noise, airborne equipment operating noise, prompt tones, alarm tones, and the superposition of one or more of the following audio signals.

[0034] In one embodiment, the alarm voice includes voice alarm sounds, the airborne equipment operating noise includes landing gear retraction and extension sounds, flap retraction and extension sounds, aircraft engine sounds, and tire sounds when skidding across the runway, and the environmental noise includes external weather sounds and air conditioning noise.

[0035] In one embodiment, the sound simulation subsystem is connected to the aircraft simulation subsystem and is used to: acquire simulation data from the aircraft simulation subsystem; parse aircraft status signals and a first excitation signal from the simulation data; and select corresponding sounds based on the aircraft status signals and the first excitation signal for superposition to simulate the sounds that the cockpit can perceive during aircraft flight.

[0036] In one embodiment, the sound simulation subsystem is also connected to the teaching control subsystem and is used to: acquire control signals sent by the teaching control subsystem; and control the operating status of the sound simulation subsystem according to the control signals of the teaching control subsystem.

[0037] In one embodiment, the sound simulation subsystem is further configured to: acquire a second excitation signal sent by the teaching control subsystem; and select a corresponding sound based on the second excitation signal to superimpose it with the sound that the cockpit can perceive during the flight of the aircraft, so as to realize sudden training during the teaching process.

[0038] In one embodiment, the first excitation signal includes an alarm excitation signal and a navigation excitation signal, and the aircraft status signal includes an aircraft flight status signal, an aircraft engine status signal, and flap and landing gear status signals.

[0039] In one embodiment, the control signal includes adjusting the volume, configuring the sound channel, selecting the sound effect, starting, freezing, and resetting, and the second excitation signal includes a thunder excitation signal, a rain excitation signal, a snow excitation signal, and a strong wind excitation signal.

[0040] In a second aspect, the present invention also provides a fault simulation training method for an aircraft cockpit simulation system as described in one or more of the preceding embodiments, comprising: receiving training content set by an instructor, the training content including training subjects and a training environment; loading aircraft parameters under a corresponding configuration according to the training subjects and the training environment to simulate the aircraft operating state corresponding to the training subjects; obtaining a selection of possible faults for each subsystem of the aircraft; setting faults for each subsystem of the aircraft according to the selection to simulate the fault operating state of the aircraft; receiving training operations from a trainee regarding the training content and the fault operating state of the aircraft; driving a model to complete logical execution according to the training operations, and outputting the execution results to complete the fault simulation training.

[0041] In one embodiment, the fault includes a rolling shutter effect in the video image display, the selection of possible faults for various aircraft subsystems includes the strength of the rolling shutter effect, and the setting of faults for various aircraft subsystems according to the selection to simulate the fault operation state of the aircraft includes: obtaining the strength of the rolling shutter effect selected by the instructor; simulating the rolling shutter effect in the video image according to the strength of the rolling shutter effect selected by the instructor to obtain simulation results, the simulation results including the vibration of the aircraft caused by the influence of the external environment during flight; and outputting the simulation results to the simulated cockpit for display.

[0042] In one embodiment, simulating the jelly effect in a video image based on the strength of the jelly effect selected by the instructor to obtain a simulation result includes: acquiring a standard image frame; calculating all vector differences between the motion vectors of each category region in the current image with the jelly effect and the standard image frame; calculating the weight of the corresponding vector difference based on the number of pixels in each category region, and performing a weighted summation of the vector difference and the corresponding weight to obtain the vector difference of the entire image; adjusting the magnitude of the vector difference corresponding to the image based on the strength of the jelly effect selected by the instructor to adjust the display effect of the jelly effect in the image.

[0043] In one embodiment, obtaining the standard image frame includes: extracting the corresponding standard image frame from the current image with the jelly effect using optical flow or jelly effect restoration software.

[0044] According to the present invention, by transmitting the simulated video signals and hard-wired signals generated by the simulation server via a bus, the video signals are transmitted together with other signals on the bus, resulting in stronger synchronization and more realistic and reliable operation. Meanwhile, the audio signals generated by the simulation server are transmitted directly to the audio equipment via a separate cable. The resulting appropriate delay more closely resembles the pilot's experience during simulated flight, effectively improving the simulation effect of flight training. Furthermore, by deploying corresponding subsystems in each part, composed of software and / or hardware, the composition of the aircraft cockpit simulation system can be refined, effectively enhancing the training effect for pilots during simulation training. Attached Figure Description

[0045] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0046] Figure 1 This is a schematic diagram illustrating an aircraft cockpit simulation system according to an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram illustrating the subsystem composition of an aircraft cockpit simulation system according to an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram illustrating an aircraft simulation subsystem according to an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram illustrating the interconnection relationships of a flight management system according to an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram illustrating the interconnection relationship of a central maintenance system according to an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram illustrating the interconnection relationship of an inertial navigation system according to an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram illustrating the top control panel according to an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram illustrating a central control panel according to an embodiment of the present invention;

[0054] Figure 9This is a schematic diagram illustrating the left operating console according to an embodiment of the present invention;

[0055] Figure 10 This is a schematic diagram illustrating the right control panel according to an embodiment of the present invention;

[0056] Figure 11 This is a schematic diagram illustrating the operating mechanism according to an embodiment of the present invention;

[0057] Figure 12 This is a schematic diagram illustrating the interconnection relationship between the landing gear system and other systems according to an embodiment of the present invention; and

[0058] Figure 13 This is a flowchart illustrating a fault simulation training method according to an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0061] The aircraft cockpit simulation system proposed in this solution is a hardware-in-the-loop simulation system that can be designed based on the cockpit layout of a specific aircraft type. It can be used to conduct various types of training and instruction, including cockpit area inspection of the aircraft's electromechanical systems, cockpit component identification, system cockpit display function checks, system power-on checks, fault diagnosis, and fault isolation. Based on this, the aircraft cockpit simulation training system of this invention can be used to complete off-site maintenance training for electromechanical and avionics professionals. In some application scenarios, the practical subjects that this aircraft cockpit simulation training system can perform include, but are not limited to, the following:

[0062] Engine start-up, APU start-up test, engine test, engine cold run, landing gear retraction and extension check, engine anti-icing system operation check, ram air turbine (RAT) hydraulic system performance check, cockpit pressurization control system manual control function check, fuel system supply and delivery power check, pressure refueling control simulation, normal braking function check, front wheel steering system function check, cockpit interior lighting system power check, fuel system power check, hydraulic system power check, door control system power check, environmental control system power check, anti-icing system power check, landing gear signal system power check, fire protection system power check, flight control system power check, external lighting power check, emergency evacuation lighting system power check, electromechanical management system central maintenance MBIT and fault information viewing, air data subsystem operation test, etc.

[0063] Figure 1 This is a schematic diagram illustrating an aircraft cockpit simulation system according to an embodiment of the present invention.

[0064] like Figure 1 As shown, the aircraft simulation system includes a simulated cockpit, a teaching terminal, a simulation server, an interface manager, and audio equipment.

[0065] The simulated cockpit can be used to simulate a cockpit and allow trainees to perform simulated operations. In some embodiments, the simulated cockpit can provide a hardware support structure for the simulated cockpit and base, simulating the spatial layout of the cockpit floor where the pilot and co-pilot reside and the equipment inside the cockpit. The simulated cockpit can be equipped with a top control panel, a front control panel, a central control panel, a left control panel, a right control panel, etc., designed with reference to the layout inside the cockpit. The size and position can be roughly consistent with the aircraft. The internal simulated equipment is arranged at a 1:1 scale. Except for some specific equipment (such as the control stick, control wheel, pedals, throttle lever, etc.) which are implemented in the form of physical simulation, other equipment can be implemented in the form of a touch screen and equipment simulation software interface.

[0066] The teaching terminal is used by instructors to publish and set up training content. It can support the development of control pages for training subjects and environments, as well as interface-driven development and process monitoring software. Examples of functions include setting training subjects, training environments, process status monitoring, and initial parameter settings for aircraft system models.

[0067] The simulation server can provide models of various aircraft components. Development on the aircraft simulation server can be based on a simulation model of a specific aircraft type. Using the aircraft's system architecture as a foundation, it analyzes the working principles and characteristics of systems and accessories, and employs digital simulation and data processing techniques to simulate the working principles and processes of each aircraft system. During the system design phase, simulation software from different fields can be used to build models of different subsystems within the aircraft system (such as engines, hydraulics, fuel, landing gear, anti-icing, and flight dynamics). Through co-simulation technology, the models built by the simulation software are effectively integrated, enabling collaborative simulation and optimization design, and dynamic simulation analysis. This fully considers the inter-system interactions and optimizes the design parameters of each aircraft subsystem to achieve the goal of global optimization of the aircraft system, providing a basis for the design of the aircraft integrated management system.

[0068] Furthermore, the initial state parameters of the aircraft system and special situation simulation model can be set to conduct special situation training.

[0069] A serial bus connects the teaching terminal and simulation server, and connects to the simulated cockpit via an interface manager. The serial bus facilitates data exchange between the teaching terminal, simulation server, and simulated cockpit. In some embodiments, the interface manager can acquire, excite, and transmit control signals from the cockpit's top control panel, and condition, adapt, and transmit electrical signals from devices such as the control stick and control wheel, thereby ensuring the real-time performance and effectiveness of the aircraft cockpit simulation system during training. In one application scenario, the simulated cockpit communicates with the interface manager via video signals, Ethernet signals, and hardwired signals. Control boxes and trackballs within the cockpit can output Ethernet signals, which are then transmitted to the serial bus via the interface manager.

[0070] The serial bus can be a 1394 bus, enabling clock synchronization and simulation scheduling. Since the system employs a real-time distributed simulation architecture, its synchronization clock uses a 1394 network clock, propagated at its base frequency through the data transmission network. Each simulation model can determine whether to perform a simulation based on its own solution cycle and the received clock signal. Therefore, the real-time performance of this system is jointly determined by the accuracy of the master clock, the characteristics of the data network transmission, and the operating system.

[0071] Specifically, the simulation server transmits the simulated video signal to the simulator cockpit via the interface manager, and the simulation server also transmits the simulated audio signal to the audio equipment, which is used in conjunction with the simulator cockpit. In one application scenario, the simulated video signal is transmitted via a bus, while the simulated audio signal is transmitted and played separately. This method distinguishes signals with different synchronization requirements. The simulated video signal on the bus, along with Ethernet and hardwired signals, has better synchronization, allowing for a more realistic user experience. The audio signal, however, allows for a certain delay, which is closer to the real feeling of a pilot during simulated flight; the audio signal is relatively slower than the video signal. Based on this, the audio signal can be transmitted independently, effectively saving bus resources and reducing transmission pressure, while also making the simulation more realistic and improving the training experience for trainees.

[0072] In practice, instructors can use the teaching terminal to control the power-on / off of the entire system's equipment, the start-up, stop, and reset of the simulation system, and complete the training environment setup and initial parameter settings for the aircraft system model. They can also monitor the status during training. Trainees complete the training by operating the simulation equipment in the simulator cockpit. Operation signals are sent to the simulation server via the interface manager. The simulation server then drives the electromechanical, avionics, and flight control models based on the trainee's operations, executing the corresponding logic and generating the corresponding result status. The result status of the aforementioned model execution is transmitted to the corresponding status indication signal in the simulator cockpit via the interface manager and audio equipment, completing the training process.

[0073] The above combination Figure 1 The basic architecture and working principle of the aircraft cockpit simulation system in this invention have been explained. The following sections will elaborate on the different components of each part.

[0074] Figure 2 This is a schematic diagram illustrating the subsystem composition of an aircraft cockpit simulation system according to an embodiment of the present invention.

[0075] like Figure 2 As shown, the simulated cockpit is equipped with a simulated cockpit subsystem, the interface manager is equipped with an interface management subsystem, the simulation server is equipped with a software integration and environment operation subsystem, a sound simulation subsystem, and an aircraft simulation subsystem, and the teaching terminal is equipped with a teaching control subsystem.

[0076] The software integration and environment operation subsystem is used to schedule the various subsystems, control the data interaction between the subsystems, and control the communication between the interface management subsystem and the aircraft simulation subsystem. In some embodiments, the main functions of the software integration and environment operation subsystem may include, for example, the following aspects:

[0077] a) It can add / delete network computing nodes and monitor the status of computing nodes in each simulation subsystem;

[0078] b) Remotely deploy the latest simulation software or models of each subsystem to each computing node;

[0079] c) Remotely run the models or software of each simulation node and monitor the running status of all simulation models;

[0080] d) Clock synchronization and data transmission are achieved using a 1394 network, with clock jitter less than 5ms;

[0081] e) Real-time monitoring and simulation of network data transmission.

[0082] The simulated cockpit subsystem supports the hardware of the simulated cockpit, converting the trainee's operations into hardwired or Ethernet signals for transmission to the interface management subsystem, and receiving video signals from the interface management subsystem for image display. Specifically, this simulated cockpit subsystem can provide a realistic aircraft pilot simulator cockpit. The cockpit dimensions and internal layout are identical to the simulated aircraft, and it can simulate instruments, displays, indicators, lights, switches, buttons, and control devices. Its appearance, dimensions, installation location, signage, functions, operating limits, instrument operating modes, operating methods, and operating characteristics can be identical to those of the aircraft. In one application scenario, the simulated cockpit subsystem may include a top control panel, a front control panel, a central control panel, a left control panel, a right control panel, and control mechanisms, including a control stick, control wheel, pedals, and throttle lever.

[0083] In one application scenario, using an aircraft cockpit as the simulation object, the simulated cockpit subsystem can include a simulated cockpit body, control mechanism functions (including a joystick, steering wheel, and pedals), equipment control boxes / panels (including an instrument panel console, overhead console, center console, left console, and right console), and other auxiliary equipment (including seats, headsets, etc.). The aforementioned components within the aircraft cockpit can be simulated using physical replicas or via a touchscreen as their hardware control interface to achieve equipment simulation. For example, the joystick, steering wheel, and pedals are simulated using physical replicas. Other panel devices use a touchscreen as their hardware control interface, and the hardware simulation is achieved by developing a software interface for the equipment control boxes / panels on the touchscreen. The structural configuration of each part will be described below. Figures 4 to 8The exemplary solution is described in detail below. Furthermore, to facilitate maintenance and future modifications, the aforementioned cabin and control box, among other components, can be designed modularly, while ensuring reliability and simulation accuracy as much as possible. Simultaneously, hard-wired signals generated by the aforementioned operating mechanisms, etc., can be transmitted via Ethernet communication to improve data transmission efficiency, reliability, and maintainability.

[0084] The aircraft simulation subsystem is used to simulate at least one component of an aircraft. It is the core of the aircraft cockpit simulation system, reflecting the realism and training effectiveness of the system's training process. The aircraft simulation subsystem can be developed based on the system's working principles; for example, simulation models of the aircraft's power plant, auxiliary power plant, fuel system, hydraulic system, and environmental control system can be developed. This subsystem provides a graphical environment, allows for the definition of parameters such as aircraft takeoff weight, fuel quantity, and center of gravity, and provides typical aircraft flight dynamics models to be embedded into the simulation system for collaborative simulation with the models mentioned earlier and later, providing environmental data stimuli for each model.

[0085] A sound simulation subsystem is used to simulate at least one sound during flight. In some embodiments, the sound simulation subsystem can simulate the sounds that flight personnel can perceive in the cockpit during various phases of an aircraft's flight, including ground taxiing, takeoff, flight, descent, and approach / landing. These sounds include ambient noise, onboard equipment operating noise, prompts, and alarms. In practical applications, a sound database stores various sounds during aircraft flight. This sound simulation subsystem can receive data from a simulation network (e.g., simulation data from an aircraft simulation system, control commands input from a teaching control subsystem), obtain a list of sounds to be played through data parsing and logical processing, then retrieve the corresponding sound files from the sound database according to the list for playback, and finally play them through an audio device. The played sound can be a single sound or a superposition of multiple sounds, thereby simulating sounds in a real environment.

[0086] In some embodiments, the sound simulation subsystem described above can be used to simulate at least one sound during flight. These sounds may include: ambient noise, airborne equipment operating noise, alert tones, alarm voices, and a superposition of one or more of the following: ambient noise, airborne equipment operating noise, tire noise, and audio signals. Alarm voices include voice alarm sounds; airborne equipment operating noise includes landing gear retraction / extension sounds, flap retraction / extension sounds, aircraft engine sounds, and tire noise as they dive across the runway; ambient noise includes external weather sounds and air conditioning noise.

[0087] In some embodiments, the sound simulation subsystem is connected to the aircraft simulation subsystem and can be used to acquire simulation data from the aircraft simulation subsystem. Then, the aircraft status signal and the first excitation signal are parsed from the simulation data. Corresponding sounds are selected and superimposed based on the aircraft status signal and the first excitation signal to simulate the sounds perceived by the cockpit during aircraft flight.

[0088] Furthermore, the aforementioned sound simulation subsystem, connected to the teaching control subsystem, is also used to: acquire control signals sent by the teaching control subsystem; and control the operating status of the sound simulation subsystem based on the control signals from the teaching control subsystem.

[0089] In some embodiments, the sound simulation subsystem is further configured to: acquire a second excitation signal sent by the teaching control subsystem; and select a corresponding sound based on the second excitation signal to superimpose it with the sound perceived by the cockpit during aircraft flight, so as to achieve sudden training during the teaching process.

[0090] Specifically, the first excitation signal includes alarm excitation signals and navigation excitation signals; the aircraft status signals include aircraft flight status signals, aircraft engine status signals, and flap and landing gear status signals; control signals may include adjusting volume, configuring sound channels, selecting sound effects, starting, freezing, and resetting; and the second excitation signal includes thunder excitation signals, rain excitation signals, snow excitation signals, and strong wind excitation signals.

[0091] The teaching control subsystem is used to set training content. In some embodiments, the teaching control subsystem is used to set training content, including setting the initial parameters of at least one component simulation model of the aircraft. For example, it can perform functions such as training environment settings (e.g., airport condition settings, flight route environment settings, meteorological condition settings, activity target settings, etc.), training subject settings (e.g., training aircraft settings, flight mission settings, fault settings, fault clearing, etc.), interface settings (e.g., audio communication settings, frequency settings, and tower settings), process status monitoring (aircraft data monitoring, control data monitoring, motion data monitoring, interface data monitoring, etc.), initial parameter settings for the aircraft system model (aircraft weight settings, start / stop, freeze / thaw, reset, etc.), record storage, and student assessment and evaluation (e.g., student information management, manual assessment, and system maintenance management). In one application scenario, the teaching control subsystem software can provide operators with settings for airport conditions, flight routes, and natural environment before the simulation environment runs. Specific settings include: providing airport selection, airport elevation, magnetic difference, takeoff direction, initial parking settings, radio frequency, field pressure, field temperature, audio signals, and other airport condition settings.

[0092] Furthermore, the teaching control subsystem can also configure potential faults in various aircraft subsystems. The fault page is divided according to ATA chapters (Chinese-English bilingual), providing both immediate and preset fault settings for each system. When a fault is selected, a pop-up text box appears in the teaching control subsystem, indicating the system that may cause the fault (e.g., left or right). Pre-selected criteria, fault effects, and descriptions are also displayed in the pop-up box. Instructors can view the fault effects and descriptions of the selected system causing the fault. The fault clearing function can also restore the aircraft to normal training status.

[0093] The interface management subsystem is used for data conversion and transmission. It includes a signal processing and interface adaptation unit and a system simulation scheduling software module. The interface management subsystem is the central hub for data conversion and transmission in the training system, responsible for the conditioning, transmission, and adaptation of data between the aircraft simulation training system and the cockpit hardware interface. The signal processing and interface adaptation unit is used to acquire electrical signals from the cockpit top control panel, front control panel, center console, left console, and right console, as well as drive light signals. The electrical signals on the cockpit console primarily include discrete quantities, analog quantities, and some bus signals. In some embodiments, through the above connection method, the interface management subsystem is used to acquire, excite, and convert data for the top control panel, front control panel, center console panel, left console, and right console in the simulated cockpit subsystem. The interface management subsystem is also used to condition, adapt, and transmit the electrical signals of the control mechanisms in the simulated cockpit subsystem.

[0094] Due to the numerous equipment modules within the cockpit, each control board contains various electrical and drive signals. Therefore, this system treats each control board as a basic unit for signal processing and interface adaptation, interacting with the aircraft system simulation subsystem and teaching control subsystem via network signals. Simultaneously, the avionics simulation portion of the virtual cockpit requires simulation image fusion functionality; therefore, the signal processing and interface adaptation unit needs to acquire video signals, transmit them, and display them on the devices. Network signals are used for acquiring and exchanging control data; devices periodically upload their status to the interface computer via the network, and the interface computer sends control signals to each device via the network.

[0095] Figure 3 This is a schematic diagram illustrating an aircraft simulation subsystem according to an embodiment of the present invention.

[0096] like Figure 3As shown, the aircraft simulation subsystem includes integrated processing software and a DDS soft bus. The integrated processing software interacts with component simulation models, algorithm models, and flight dynamics models via the DDS soft bus. The DDS-based distributed real-time simulation soft bus effectively connects various simulation software programs on different computers, enabling unified simulation management, including reliable data communication and time-tracking mechanisms. It also allows for sharing interface data between different simulation software programs to achieve system simulation analysis. In some embodiments, the component simulation models may include one or more of the following: hydraulic system model, environmental control system model, fuel system model, power plant model, auxiliary power plant model, oxygen system model, environmental protection system model, control model, and lighting system model. The control models include landing gear control system model and door control system model.

[0097] In some embodiments, the above-mentioned aircraft simulation subsystem can be used to run simulation models of at least one component of an aircraft, including: receiving control signals from the simulated cockpit subsystem and initial parameters of the model from the teaching and control subsystem, running the corresponding setup simulation model, and outputting data to other subsystems, wherein the control signals are generated by the trainee through operating the simulated cockpit.

[0098] In some embodiments, the simulation interface may include an AMESim interface, a Simulink interface, and a FlightSim interface. The component simulation model is connected to the DDS soft bus via the AMESim and Simulink interfaces, the algorithm model is connected to the DDS soft bus via a C / C++ interface, and the flight dynamics model is connected to the DDS soft bus via the FlightSim interface.

[0099] The integrated processing software includes flight management software, central alarm software, and central maintenance software.

[0100] Flight management software primarily performs functions such as aircraft navigation, flight plan management, performance calculation, trajectory optimization, guidance functions, airdrop mission calculation, database management, comprehensive monitoring, and alarms. Its main tasks are to optimize flight trajectories, improve navigation accuracy, and reduce pilot workload, thereby ensuring high-efficiency mission completion. The flight management software utilizes navigation and aircraft status data input from sensors, and based on reference data provided by navigation and performance databases, to perform real-time flight guidance calculations, assisting pilots in controlling the aircraft's flight trajectory and ensuring it flies according to the pre-defined flight plan and the currently selected performance mode.

[0101] The Flight Management System (FM) model within the integrated processing software consists of an application software (FMSA) running on the Common Processing Module (CPM) within the Integrated Processor (IPC) and other subsystems providing functional support. The FMS uses the Analog Control Display Unit (SCDU), Navigation Display (ND), Primary Flight Display (PFD), and keyboard provided by the Display and Control Subsystem (CDS) as the primary human-machine interface; inertial / satellite integrated navigation equipment (INS), air data equipment (ADC), radio navigation equipment, and electromechanical management computer (EMP) as navigation sensors and aircraft status sensors; and the Automatic Flight Control System (AFCS) as the main flight execution component. Furthermore, the FMSA updates the navigation and performance databases through loading / unloading devices and can unload pilot database content.

[0102] The flight management system model supports pilots in completing the entire flight mission, including guidance and calculations of information related to the flight process. Specifically, this flight management system model can simulate the following functions: integrated navigation management, flight plan management, performance calculation, guidance, military mission management, database management, and integrated monitoring and alarms. Correspondingly, based on functional division, this flight management system model can be divided into the following types of interfaces: integrated navigation control interface, flight plan editing interface, performance setting interface, control interface (suppression, conversion, and interception, etc.), and database interface.

[0103] Based on the above interface settings, the inter-interface relationships of the flight management system functions, as well as the functional division and dependencies between functions of the flight management system, are as follows: Figure 4 As shown:

[0104] a) Flight plan editing and management functions are completed through the flight plan editing interface. This relies on the navigation database management function, which supports other functions through the database interface. b) Horizontal flight control is set up through the control interface. Horizontal flight control provides horizontal guidance based on aircraft parameters provided by the flight plan and integrated navigation functions. c) Integrated navigation calculates aircraft position, speed, altitude, and other parameters. d) Track planning generates a three-dimensional flight track based on the current progress. e) Basic aircraft performance calculation determines the performance parameters of the aircraft platform under various conditions. f) Vertical flight control controls the aircraft altitude to meet the vertical constraints of the flight plan. g) Based on the calculated three-dimensional flight track, HSD information and fuel arrival time information can be calculated. h) Interface symbols represent the control that CDS can exert over the execution of a specific function.

[0105] The central alarm software receives fault alarm information from various systems / equipment inside the aircraft, configuration alarm information from the aircraft itself, and threat alarm information from outside the aircraft (hazard level, warning level, attention level, advisory level, alert level). It performs logical processing and priority sorting of alarm information, drives light alarms, drives the display processing unit to display alarm information, and drives audio equipment to generate voice / tone alarms. The types of alarms received and processed include threat alarms output by electronic support and reconnaissance equipment, ground proximity alarm equipment, etc., fault alarms output by internal systems / equipment such as flight control, engine, hydraulic, and environmental control, and aircraft configuration alarms.

[0106] The central maintenance software implements basic functions such as processing, storing, retrieving, displaying, and calling logic for aircraft system fault information, ground testing operation procedures, status monitoring operation procedures, hardware and software configuration management and identification, and data loading operation procedures.

[0107] The central maintenance system can be constructed using the aforementioned central maintenance software, and its interconnection with other systems is as follows: Figure 5 As shown, the central maintenance system software can display maintenance information using the left and right multi-function flight displays and transmit data to external systems via printer or data link. It can also connect to avionics and non-avionics systems, such as receiving LRU (Line Replaceable Unit) faults from systems like air conditioning, autopilot, fire protection, fuel systems, landing gear, lighting, navigation, oxygen, auxiliary power systems, and engine systems, and displaying the name of the faulty LRU.

[0108] The aircraft simulation subsystem of this invention also includes an inertial navigation system model (hereinafter referred to as the inertial navigation system model). The inertial navigation system is the parameter calculation system of the trainer, realizing an all-weather, all-attitude, autonomous navigation system, with alignment, navigation, and navigation data output functions. The system model can receive the raw acceleration and angular velocity output from the aircraft equations, receive satellite information and atmospheric data information from the GPS subsystem, and output the aircraft's position, heading, attitude, velocity, acceleration, angular velocity, altitude, global magnetic difference, and time information through the HB6096 simulation interface for use in aircraft navigation calculations, data display, flight control, parameter recording, and airdrop / parachuting.

[0109] like Figure 6As shown, this inertial navigation system can be connected to the main flight control system, the automatic flight control system, the remote data concentrator (RDC), and the distributed processing unit (DPU). The inertial navigation system may include inertial satellite navigation system 1, inertial satellite navigation system 2, and inertial satellite navigation system 3. The two inertial satellite navigation systems mentioned above are interconnected with the main flight control system, the automatic flight control system, the RDC, and the DPU, respectively.

[0110] External systems interconnected with the aforementioned engine system (model) can include models of flight control systems, avionics systems, power supply systems, hydraulic systems, environmental control systems, fuel systems, and fire protection systems.

[0111] Furthermore, the aforementioned aircraft simulation subsystem is also used to run an atmospheric data model, which provides flight parameters calculated based on atmospheric data. In some embodiments, the atmospheric data model is also used to receive corresponding parameters set by the teaching and control subsystem and sensor information calculated by the aircraft simulation subsystem, and then calculate flight control parameters, which are further processed by the aircraft simulation subsystem. The atmospheric data model may include the following software modules: a data controller, a model controller, a data processor, and a sensor controller. The atmospheric data model receives the ambient temperature set by the teaching and control subsystem and the total pressure, static pressure, angle of attack, and sideslip angle information calculated by the aircraft model in the flight system. After passing through the sensor model, this data is sent as input to the atmospheric data computer simulation module. Through various stages, it obtains vacuum speed, indicated airspeed, Mach number, barometric altitude, corrected barometric altitude, vertical velocity, etc., and finally sends this data to various flight dynamics models for calculation to display flight data.

[0112] The above provides a detailed explanation of the software simulation components of each simulation subsystem, and illustrates how the aircraft simulation subsystem achieves structural simulation through the coordination of various functional models. The following section will elaborate on the hardware and software panels of the simulated cockpit subsystem.

[0113] In some embodiments, each control board, panel, and console in the simulated cockpit subsystem is implemented using physical simulation components and / or virtual simulation interfaces.

[0114] like Figure 7As shown, the top control panel may include one or more of the following control panels: emergency positioning control panel, cabin lighting control panel, avionics start control panel, flight control system control panel, hydraulic system control panel, backup shutdown control panel, power system control panel, engine start control panel, fuel system control panel, external lighting control panel, windshield wiper control switch, landing lighting control panel, fire protection system control panel, cabin voice monitoring control panel, electromechanical management system control panel, lifesaving system control panel, oxygen system control panel, anti-icing system control panel, gas supply system control panel, air conditioning system control panel, and pressure regulation system control panel.

[0115] The forward control panel includes one or more of the following: left warning light, right warning light, left display control panel, right display control panel, automatic flight control panel, approach warning light, landing gear control handle, landing gear light, and automatic brake selector panel.

[0116] like Figure 8 As shown, the central control console includes one or more of the following: emergency stop brake, horizontal stabilizer position indicator, multifunction display, horizontal stabilizer trim control handle, speed brake handle, horizontal stabilizer trim cut-off switch, flap handle, left and right trackballs, left and right multifunction keyboards, normal stop switch, flap and slat overrun control panel, radio tuning unit, fire suppression control panel, throttle control console, audio control panel, trim control panel, selection switch panel, cabin door control panel, cockpit airdrop control panel, and electronic warfare control panel. The emergency stop brake, horizontal stabilizer position indicator, multifunction display, horizontal stabilizer trim control handle, speed brake handle, flap handle, left and right trackballs, and left and right multifunction keyboards can all be implemented using physical simulation components and are arranged around the multifunction display. The other components are simulated through a software interface on a touchscreen.

[0117] like Figure 9 As shown, the left control panel includes the driver's side oxygen mask control panel, left dimming control panel, driver's side front wheel steering handle, left head-up display control panel, key control box, time key control box, and headphone / microphone jack assembly. The headphone / microphone jack assembly is implemented using a physical simulation component, while the others can be simulated through a software interface on the touchscreen.

[0118] like Figure 10 As shown, the right control panel includes the passenger-side oxygen mask control, right dimming control panel, passenger-side front wheel steering handle, right head-up display control panel, oxygen shut-off valve switch, and task loading / unloading card. Additionally, it includes a headset / microphone jack assembly for providing voice communication functionality for the passenger side.

[0119] like Figure 11 As shown, the cockpit also includes a control stick mechanism, a control wheel mechanism, and a foot pedal mechanism, all of which can be simulated using physical simulation components.

[0120] In some embodiments, the present invention also simulates a power supply system model that supplies power to the aircraft cockpit simulation system, including an electrical system model, a fuel system model, etc.

[0121] Based on this, the aforementioned aircraft simulation subsystem also includes: an electrical system model, used to interconnect with the hydraulic system model, fuel system model, power plant model, oxygen system model, landing gear system model, door system model, and lighting system model to achieve data interaction. The electrical system model is also used to interact with the avionics system model, flight control system model, engine model, and electromechanical management system model, and for state interconnection with the electrical control panel.

[0122] Furthermore, the aircraft simulation subsystem also includes a fuel system model, which interacts with the engine model, auxiliary power unit model, electromechanical management system model, power supply system model, teaching control subsystem, and fuel control panel. Additionally, this fuel system model is interconnected with the fuel control panel to achieve fuel control.

[0123] Under normal fuel system conditions, each AC main power channel operates independently, with the alternator supplying power only to the generator busbar and AC main busbar of its respective channel. When a channel fails, the system automatically isolates and protects it, allowing another generator on the same side to supply power to the two generator busbars and AC main busbar on that side. If both channels on the same side fail, and the auxiliary generator is not operating, the power supply control management subsystem uses its power conversion control function to allow the two generators on the other side to supply power to the two AC main busbars on this side. If the auxiliary generator is operating, it supplies power to the two generator busbars and AC main busbar on this side.

[0124] The aforementioned aircraft simulation subsystem also includes a landing gear retraction system model, which is used to interact with the landing gear control handle, hydraulic system model, power supply system model, and electromechanical management system.

[0125] like Figure 12As shown, the landing gear retraction system model's interconnections with other systems in the aircraft cockpit simulation system include: the landing gear retraction system model is connected to the landing gear indicator light box to display landing gear position information. This landing gear retraction system model can also connect to the ground proximity warning system, system control unit (SCU), brake control unit (BCU), electromagnetic pulse weapon (EMP), digital air waybill platform (DAP), throttle platform thrust reverser unlocking device, generator control unit, etc., to achieve corresponding functions based on wheel load information. The landing gear retraction system model is also connected to the landing gear control handle, hydraulic system, power supply system, and EMS to receive power signals for corresponding control signals and achieve corresponding functions.

[0126] In some embodiments, the interface manager includes an image generation computer and an interface computer. The image generation computer is connected to the front control panel and the central console via a DVI signal. The interface computer is used to connect to the front control panel, the central console, the top control panel, the left console, and the right console via a network switch. The interface manager also includes a power control box, which is connected to the front control panel, the central console, the top control panel, the left console, and the right console respectively, for providing DC power.

[0127] Figure 13 This is a flowchart illustrating a fault simulation training method according to an embodiment of the present invention.

[0128] like Figure 13 As shown, in step S1301, the training content set by the instructor is received. The training content includes training subjects and training environment. In practical applications, instructors log in to the system control and training setting software through the teaching terminal, and while completing demonstrations and explanations according to the teaching syllabus requirements, they set the training subjects and training environment, etc.

[0129] In step S1302, aircraft parameters under the corresponding configuration are loaded according to the training subject and training environment to simulate the aircraft operating state corresponding to the training subject. After receiving the training subject and training environment information set by the teaching terminal, the simulation server can adjust and simulate the aircraft operating state corresponding to the training subject by loading the various data models therein, and send corresponding control signals to the simulation cockpit to simulate the aircraft operating state.

[0130] In step S1303, a selection of possible faults for each subsystem of the aircraft is obtained. In some embodiments, the selection of faults can be set together when setting training subjects and training environment, or it can be set during training.

[0131] In step S1304, fault settings are configured for each subsystem of the aircraft according to the selected parameters to simulate the aircraft's faulty operating state. Each subsystem in the simulation server can inject corresponding fault information into the aircraft simulation system according to the configured faults, thereby simulating the aircraft under that fault state.

[0132] In step S1305, the training operation of the trainee regarding the training content and the aircraft's fault operation status is received. During the instructor's explanation, the trainee views the working process demonstration and, according to the training subjects set by the instructor, operates the three cockpit simulation panels—the autopilot control panel, the status selection panel, and the weapon operator control panel—by viewing the content displayed in the integrated instrument software and the visual display software, completing the operation process of the training subjects. (Before and during the trainee's operation according to the training subjects, the instructor selects to inject various types of system faults through the fault analysis module, allowing the trainee to learn how to perform fault analysis and troubleshooting based on the fault phenomena.)

[0133] In step S1306, the logic execution is completed according to the training operation-driven model, and the execution result is output to complete the fault simulation training. In some embodiments, the system control and training setting software in the teaching terminal records the student's operation process data. The instructor reviews the data recorded by the software and evaluates the student's operation process.

[0134] Furthermore, trainees can log in to the system control and training settings software to view system-recorded data and instructor evaluations, and make corrections and summaries.

[0135] In some embodiments, the aforementioned faults include a rolling shutter effect in the video image display. The selection of possible faults for various aircraft subsystems includes the intensity of the rolling shutter effect. For example, in the teaching control subsystem of the teaching terminal, a selection switch and an adjustment switch for the rolling shutter effect can be set accordingly. The adjustment switch can be a rotary switch, thereby achieving stepless adjustment of the intensity of the rolling shutter effect. When setting faults for various aircraft subsystems according to the instructor's selection to simulate the aircraft's fault operation state, firstly, the intensity of the rolling shutter effect selected by the instructor is obtained. Then, the rolling shutter effect in the video image is simulated according to the intensity of the rolling shutter effect selected by the instructor to obtain simulation results. These simulation results include the vibration of the aircraft during flight caused by external environmental influences. Finally, the simulation results are output to the simulated cockpit subsystem for display.

[0136] The above simulation process can be implemented as follows: Specifically, a standard image frame is acquired. The motion vectors of each category region in the current image with the jelly effect are calculated, along with all vector differences between the standard image frame and the frame. Then, the weights of the corresponding vector differences are calculated based on the number of pixels in each category region, and the vector differences and their corresponding weights are summed in a weighted manner to obtain the vector difference for the entire image. The magnitude of the vector difference corresponding to the image is adjusted according to the strength of the jelly effect selected by the instructor, thereby adjusting the display effect of the jelly effect in the image.

[0137] Understandably, when instructors select the degree of jelly effect, this adjustment switch corresponds to the magnitude of the aforementioned vector difference. Each vector difference represents the degree of deviation between the jelly effect in the image and the standard image. Thus, by adjusting the magnitude of this vector difference, the degree of jelly effect can be selected.

[0138] In one application scenario, the jelly effect in video images can be simulated and adjusted in stages or gradually, thereby adjusting the jelly effect to a level that the learner can adapt to.

[0139] In practical applications, since the images used inherently exhibit a jelly effect, it is necessary to first extract the corresponding standard image frame—that is, a standard image without the jelly effect. In some embodiments, optical flow or jelly effect restoration software can be used to obtain the standard image frame corresponding to the current image with the jelly effect. By calculating the overall vector difference between the motion vectors of each category region in the current image and the standard image frame, the degree of the jelly effect can be adjusted using the magnitude of this overall vector difference. Specifically, the weight of the corresponding vector difference is calculated based on the number of pixels (i.e., area) of each category region in the image. The vector difference and the corresponding weights are then weighted and summed to obtain the final vector difference of the entire image. This vector difference can be used to produce a corresponding degree of jelly effect in the image, thereby achieving dynamic control of the jelly effect by adjusting the magnitude of the vector difference.

[0140] In some embodiments, the process of adjusting the jelly effect can be achieved in the following ways:

[0141] The acquired video frame images with the rolling shutter effect are binarized, and then the frame difference between consecutive images is calculated. The larger the frame difference, the more pronounced the rolling shutter effect. Since new objects may enter the image during scene movement, judging the rolling shutter effect solely by frame difference is inaccurate. Therefore, this scheme introduces nearest neighbor density. Unlike traditional methods that calculate nearest neighbor density based on distance, this scheme uses the grayscale value between pixels after calculating the frame difference as the indicator of nearest neighbor density.

[0142] Calculate the nearest neighbor density of each pixel's grayscale value in the image. If the Local Outlier Factor (LOF) score for each point is close to 1, it indicates that the local density of point p is close to its neighbors. If the LOF score is less than 1, it indicates that p is in a relatively dense region, unlike an outlier. If the LOF score is much greater than 1, it indicates that p is relatively distant from other points and is likely an outlier. Therefore, pixels can be clustered according to their grayscale values; pixels with consistent or very similar grayscale values ​​will be in similar local densities.

[0143] In the process of calculating the nearest neighbor density, each image with the same or similar gray-level connected regions is treated as a target category. That is, regions with similar gray values ​​but whose pixels are not connected are classified into different target categories.

[0144] For each target category in consecutive frames, a union is taken. Using the overall motion vectors between pixels within this union region, the motion vectors are calculated using the Enhanced Predictive Zonal Search (EPZS) algorithm. Next, similar motion vectors are merged again using k-nearest neighbor density to obtain different categories with different motion vectors. On the other hand, a standard image frame corresponding to the current image with the jelly effect is obtained using optical flow or jelly effect restoration software, and the overall vector difference between the motion vectors of each category region and the standard image frame is calculated.

[0145] Finally, the weight of the corresponding vector difference is calculated based on the number of pixels (i.e. area) in each category region. The vector difference and the corresponding weight are weighted and summed to obtain the final vector difference of the entire image. This allows for dynamic control of the jelly effect by adjusting the magnitude of the vector difference.

[0146] While various embodiments of the invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of the invention. The appended claims are intended to define the scope of the invention and therefore cover any modular compositions, equivalents, or alternatives within the scope of these claims.

Claims

1. An aircraft simulation system, characterized in that, include: A simulator is used to simulate a cockpit and allow trainees to perform simulated operations in it. The teaching terminal is used by instructors to publish and set training content. The simulation server provides models of various aircraft components. A serial bus connects the teaching terminal and the simulation server, and is connected to the simulator cockpit through an interface manager; the serial bus is used for data interaction between the teaching terminal, the simulation server, and the simulator cockpit. The simulation server transmits the simulated video signal generated by the simulation to the simulated cockpit through the interface manager, and the simulation server transmits the simulated audio signal generated by the simulation to the audio device, wherein the audio device is used in conjunction with the simulated cockpit; The training content includes training subjects and a training environment; aircraft parameters under the corresponding configuration are loaded according to the training subjects and the training environment to simulate the aircraft operating state corresponding to the training subjects; selections for possible faults in each subsystem of the aircraft are obtained; fault settings for each subsystem of the aircraft are completed according to the selections to simulate the fault operating state of the aircraft; training operations from trainees are received regarding the training content and the fault operating state of the aircraft; the training operations drive the model to complete logical execution and output the execution results to complete fault simulation training, where faults include the jelly effect in video image display; The fault includes a jelly effect in the video image display, comprising: acquiring the strength of the jelly effect selected by the instructor; simulating the jelly effect in the video image according to the strength of the jelly effect selected by the instructor to obtain a simulation result, wherein: acquiring a standard image frame; Calculate all vector differences between the motion vectors of each category region in the current image with the jelly effect and the standard image frame; calculate the weight of the corresponding vector difference based on the number of pixels in each category region, and perform a weighted sum of the vector differences and their corresponding weights to obtain the vector difference of the entire image; adjust the magnitude of the vector difference corresponding to the image according to the strength of the jelly effect selected by the instructor to adjust the display effect of the jelly effect in the image; the simulation results include the vibration of the aircraft caused by the external environment during flight; output the simulation results to the simulated cockpit for display; The acquisition of standard image frames includes: The corresponding standard image frame is extracted from the current image with the jelly effect using optical flow or jelly effect restoration software. The process of regulating the jelly effect includes: The acquired video frame images with the jelly effect are binarized, and then the frame difference between consecutive images is calculated. The larger the frame difference, the more obvious the jelly effect. Calculate the nearest neighbor density of the gray values ​​of each pixel in the image. If the outlier score (LOF) of each point is close to 1, it means that the local density of the sample point p is close to its neighbors. If the LOF is less than 1, it means that p is in a relatively dense region and does not seem to be an outlier. If the LOF is much greater than 1, it means that p is relatively distant from other points and is likely an outlier. In this way, the pixels are clustered according to their gray values, so that pixels with the same or very similar gray values ​​are in similar local densities. In the process of calculating the nearest neighbor density, connected regions with the same or similar gray levels in each image are treated as a target category. That is, regions with similar gray values ​​but unconnected pixels are classified into different target categories. The union of target categories in consecutive frames is taken, and the motion vector is calculated using the overall motion vector between pixels within the union region of the target categories through the EPZS enhanced prediction region search algorithm. Then, the same or similar motion vectors are merged again using the k-nearest neighbor density to obtain different categories with different motion vectors.

2. The aircraft simulation system according to claim 1, characterized in that, The simulated cockpit communicates with the interface manager via video signals, Ethernet signals, and hardwired signals.

3. The aircraft simulation system according to claim 2, characterized in that, The serial bus is a 1394 bus.

4. The aircraft simulation system according to claim 1, characterized in that, The simulated cockpit is equipped with a simulated cockpit subsystem, the interface manager is equipped with an interface management subsystem, the simulation server is equipped with a software integration and environment operation subsystem, a sound simulation subsystem, and an aircraft simulation subsystem, and the teaching terminal is equipped with a teaching control subsystem. The software integration and environment operation subsystem is used to schedule the various subsystems, control the data interaction between the various subsystems, and control the communication between the interface management subsystem and the aircraft simulation subsystem. The simulated cockpit subsystem is used to support the hardware of the simulated cockpit, convert the trainee's operations into hardwired signals or Ethernet signals, transmit them to the interface management subsystem, and receive video signals transmitted by the interface management subsystem for image display. The aircraft simulation subsystem is used to run simulation models of at least one component of the aircraft. The sound simulation subsystem is used to simulate at least one sound during flight; The teaching management and control subsystem is used to set training content; The interface management subsystem is used for data conversion and transmission.

5. The aircraft simulation system according to claim 4, characterized in that, The teaching management and control subsystem is used to set training content, including: The teaching control subsystem is used to set the initial parameters of simulation models of at least one component of the aircraft.

6. The aircraft simulation system according to claim 4, characterized in that, The aircraft simulation subsystem is used to run simulation models of at least one component of the aircraft, including: The system receives control signals from the simulated cockpit subsystem and initial model parameters from the teaching control subsystem, runs the corresponding setup simulation model, and outputs data to other subsystems. The control signals are generated by the student through operating the simulated cockpit.

7. An aircraft simulation system according to claim 6, characterized in that, The aircraft simulation subsystem includes integrated processing software and a DDS soft bus; the integrated processing software interacts with component simulation models, algorithm models and flight dynamics models through the DDS soft bus.

8. An aircraft simulation system according to claim 7, characterized in that, The component simulation model is connected to the DDS soft bus via the AMESim and Simulink interfaces, the algorithm model is connected to the DDS soft bus via the C / C++ interface, and the flight dynamics model is connected to the DDS soft bus via the FlightSim interface.

9. An aircraft simulation system according to claim 7, characterized in that, The integrated processing software includes flight management software, central alarm software, and central maintenance software.

10. An aircraft simulation system according to claim 4, characterized in that, The aircraft simulation subsystem is also used to run an atmospheric data model, which provides flight parameters calculated based on atmospheric data.

11. An aircraft simulation system according to claim 10, characterized in that, The atmospheric data model is used to receive the corresponding parameters set by the teaching control subsystem and the sensor information calculated by the aircraft simulation subsystem, and then calculate the flight control parameters, which are further processed by the aircraft simulation subsystem.

12. An aircraft simulation system according to claim 4, characterized in that, The simulated cockpit subsystem includes a top control panel, a front control panel, a central control panel, a left control panel, a right control panel, and a control mechanism, which includes a control stick, a control wheel, pedals, and a throttle lever.

13. An aircraft simulation system according to claim 12, characterized in that, The interface management subsystem is used to complete the signal acquisition, excitation, and data conversion and transmission of the top control panel, front control panel, central control panel, left control panel, and right control panel of the cockpit in the simulated cockpit subsystem; the interface management subsystem is also used to realize the conditioning, adaptation, and transmission of electrical signals of the control mechanism in the simulated cockpit subsystem.

14. An aircraft simulation system according to claim 12, characterized in that, The component simulation model includes one or more of the following: hydraulic system model, environmental control system model, fuel system model, power unit model, auxiliary power unit model, oxygen system model, environmental protection system model, control model, and lighting system model; the control model includes landing gear control system model and door control system model.

15. An aircraft simulation system according to claim 12, characterized in that, The top control panel includes one or more of the following control panels: Emergency positioning control panel, cabin lighting control panel, avionics start control panel, flight control system control panel, hydraulic system control panel, backup shutdown control panel, power system control panel, engine start control panel, fuel system control panel, external lighting control panel, windshield wiper control switch, landing lighting control panel, fire protection system control panel, cabin voice monitoring control panel, electromechanical management system control panel, lifesaving system control panel, oxygen system control panel, anti-icing system control panel, gas supply system control panel, air conditioning system control panel, and pressure regulation system control panel.

16. An aircraft simulation system according to claim 12, characterized in that, The front control panel includes one or more of the following: Left warning light, right warning light, left display control panel, right display control panel, automatic flight control panel, approach warning light, landing gear control handle, landing gear light, and automatic brake selector panel.

17. An aircraft simulation system according to claim 12, characterized in that, The central control console includes one or more of the following: Emergency stop brake, horizontal stabilizer position indicator, multifunction display, horizontal stabilizer trim control handle, speed brake handle, horizontal stabilizer trim cut-off switch, flap handle, left and right trackballs, left and right multifunction keyboards, normal stop switch, flap and slat overrun control panel, radio tuning unit, fire suppression control panel, throttle control panel, audio control panel, trim control panel, selection switch panel, cabin door control panel, cockpit airdrop control panel, and electronic warfare control panel.

18. An aircraft simulation system according to claim 12, characterized in that, The left control panel includes a driver's side oxygen mask control panel, a left dimming control panel, a driver's side front wheel steering handle, a left head-up display control panel, a key control box, a time key control box, and an earphone / phone jack assembly.

19. An aircraft simulation system according to claim 12, characterized in that, The right control panel includes the passenger-side oxygen mask control, the right dimming control panel, the passenger-side front wheel steering handle, the right head-up display control panel, the oxygen shut-off valve switch, and the task loading / unloading card.

20. An aircraft simulation system according to any one of claims 12-19, characterized in that, The control boards, panels, and consoles in the simulated cockpit subsystem are all implemented through physical simulation components and / or virtual simulation interfaces.

21. An aircraft simulation system according to claim 14, characterized in that, The aircraft simulation subsystem also includes: The electrical system model is used to interact with the hydraulic system model, fuel system model, power plant model, oxygen system model, landing gear system model, door system model, and lighting system model; the electrical system model is also used to interact with the avionics system model, flight control system model, engine model, and electromechanical management system model, and the electrical system model is also used for state interconnection with the electrical control panel.

22. An aircraft simulation system according to claim 21, characterized in that, The aircraft simulation subsystem also includes: The fuel system model is used to interact with the engine model, auxiliary power unit model, electromechanical management system model, power supply system model, teaching control subsystem, and fuel control panel.

23. An aircraft simulation system according to claim 22, characterized in that, The aircraft simulation subsystem also includes: The landing gear retraction system model is used to interact with the landing gear control handle, hydraulic system model, power supply system model, and electromechanical management system.

24. An aircraft simulation system according to claim 12, characterized in that, The interface manager includes: The image generation computer is connected to the front control panel and the central console via a DVI signal; An interface computer is used to connect to the front control panel, central console, top control panel, left console, and right console via a network switch.

25. An aircraft simulation system according to claim 24, characterized in that, The interface manager also includes: The power control box is connected to the front control panel, central control panel, top control panel, left control panel, and right control panel respectively, and is used for DC power supply.

26. An aircraft simulation system according to claim 4, characterized in that, The sound simulation subsystem is used to simulate at least one sound during flight, including: ambient noise, airborne equipment operating noise, prompt tones, alarm tones, and the superposition of one or more of the following audio signals.

27. An aircraft simulation system according to claim 26, characterized in that, The alarm voice includes voice alarm sounds, the airborne equipment operating noise includes landing gear retraction and extension sounds, flap retraction and extension sounds, aircraft engine sounds, and tire sounds when diving across the runway, and the environmental noise includes external weather sounds and air conditioning noise.

28. An aircraft simulation system according to claim 26, characterized in that, The sound simulation subsystem is connected to the aircraft simulation subsystem and is used for: Obtain the simulation data of the aircraft simulation subsystem; The aircraft state signal and the first excitation signal are extracted from the simulation data; The corresponding sound is selected and superimposed based on the aircraft status signal and the first excitation signal to simulate the sound that the cockpit can perceive during the flight of the aircraft.

29. An aircraft simulation system according to claim 28, characterized in that, The sound simulation subsystem is also connected to the teaching control subsystem for: Obtain the control signals sent by the teaching management and control subsystem; The operating status of the sound simulation subsystem is controlled according to the control signals of the teaching control subsystem.

30. An aircraft simulation system according to claim 29, characterized in that, The sound simulation subsystem is also used for: Obtain the second excitation signal sent by the teaching control subsystem; Based on the second excitation signal, the corresponding sound is selected and superimposed with the sound that can be perceived by the cockpit during the flight of the aircraft to realize sudden training during the teaching process.

31. An aircraft simulation system according to claim 29, characterized in that, The first excitation signal includes an alarm excitation signal and a navigation excitation signal, and the aircraft status signal includes an aircraft flight status signal, an aircraft engine status signal, and flap and landing gear status signals.

32. An aircraft simulation system according to claim 30, characterized in that, The control signals include adjusting the volume, configuring the sound channel, selecting the sound effect, starting, freezing, and resetting. The second excitation signal includes a thunder excitation signal, a rain excitation signal, a snow excitation signal, and a strong wind excitation signal.

Images