A flight simulator virtual instructor training module

Abstract

A flight simulator virtual instructor training module that interfaces with a home flight simulator is provided. The system and method aims to enhance the flight simulation experience by providing real-time feedback and scoring based on the user's performance, leveraging artificial intelligence to generate context-specific messages and audio feedback. This could potentially improve the user's learning curve and overall flight simulation experience.

Description

A FLIGHT SIMULATOR VIRTUAL INSTRUCTOR TRAINING MODULECROSS-REFERNCE TO RELATED APPLICATIONS

[0001] The present application is a non-provisional application of United States Provisional Application No. 63 / 553,847 filed February 15, 2024 the entirety is hereby incorporated by reference for all purposes.TECHNICAL FIELD

[0002] The present invention relates to flight simulation, and more particularly, to a system and method for providing a flight simulator training model with generative feedback in a home flight simulator.BACKGROUND

[0003] Flight simulators are computer-based systems that simulate the operation and environment of an aircraft for various purposes, such as training, entertainment, or research. Flight simulators can provide realistic and immersive experiences for users, who can control various aspects of the simulated flight, such as the aircraft model, the flight plan, the weather, and the cockpit instruments.

[0004] However, flight simulators may not provide adequate feedback or guidance for users who wish to improve their flight skills or prepare for real-world flight tests. For example, flight simulators may not detect or correct errors or deviations from the expected flight performance, such as improper speed, altitude, attitude, or navigation. Moreover, flight simulators may not provide interactive and adaptive instruction or evaluation for users who wish to learn or practice specific flight exercises, such as takeoffs, landings, stalls, or emergency procedures.

[0005] Increasing graphical and computing capacity of personal computers have afforded pilots the ability to run ultra-realistic training from home. However, learning to fly a simulated aircraft can be challenging and frustrating without proper guidance and feedback. Moreover, hiring a human instructor certified forflight training or joining a flight school can be costly and time-consuming. Therefore, there is a need for a flight simulator virtual instructor training module that can interface with a home flight simulator and provide realistic, adaptive, and personalized instruction and feedback to the user based on their performance and progress in a way that follows certified flight training standards.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure. FIG. 1 is a block diagram of an exemplary system for providing a flight simulator virtual instructor training module interfacing with a home flight simulator, according to one embodiment of the disclosure.FIG. 2 is a system diagram for providing a flight simulator virtual instructor training module interfacing with a home flight simulator, according to one embodiment of the disclosure.FIG. 3 is a method diagram of an exemplary flight exercise maneuver and associated flight triggers, according to one embodiment of the disclosure.FIG. 4 is a method diagram of pausing and resetting the flight simulator, according to one embodiment of the disclosure. FIG. 5 is a method diagram of determining a score for the user of the flight simulator, according to one embodiment of the disclosure.DETAILED DESCRIPTION

[0007] The present disclosure relates to a method and system for providing a flight simulator virtual instructor training module interfacing with a home flight simulator. The method and system aim to enhance the user's learning experience and performance by providing a virtual instructor that can instruct the user during the flight simulation, using natural language processing and machine learning techniques.

[0008] A system and method for providing a flight simulator virtual instructor training module that interfaces with a home flight simulator is provided. The system includes a processor configured to execute instructions stored in a memory to perform various tasks. These tasks include receiving a flight exercise selection from a user in the home flight simulator, monitoring flight triggers associated with the flight exercise maneuver, and receiving flight data from the home flight simulator related to the user's performance. The system determines if an exception has occurred when flight data variables exceed the flight triggers, assesses the severity and style of the exception, and provides context feedback. It also determines a score based on the received flight data compared to ideal flight data, pauses and resets the flight simulator, and commands control of the flight simulator to return to a safe condition and demonstrate the correct technique to the user. Additionally, the system generates a context message identifying relevant variables exceeding flight triggers, provides the context message to a generative artificial intelligence (GAI), converts interaction text from the GAI to audio output, and delivers audio feedback to the user to provide verbal feedback on the exception by the virtual instructor.

[0009] Further, a system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. One general aspect includes a computer-implemented method of providing an aircraft flight simulator virtual instructor training module interfacing with a home flightsimulator executed on a computing device. The computer - implemented method also includes receiving a flight exercise selection from a user in the home flight simulator; monitoring flight triggers associated with the flight exercise maneuver, receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated the flight exercise, determining that an exception has occurred when variables received in flight data exceed the flight triggers, determining severity and style of the exception occurring and producing context feedback relevant to the severity, determining a score associated with the received flight data compared to ideal flight data relative to the flight triggers, pausing and resetting the flight simulator, commanding control of the flight simulator to return to a safe condition and demonstrate technique to the user, generating a context message identifying relevant variables exceeding flight triggers, providing the context message to a generative artificial intelligence (gai), converting interaction text received from the gai to audio output, and delivering audio output to the user to provide verbal feedback on the exception to the user by the virtual instructor. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0010] Implementations may include one or more of the following features. The method where subsequent to pausing and resetting the flight simulator the method may include: taking control of the aircraft in the flight simulator and demonstrating proper form to the user based on the flight exercise maneuver based upon the ideal flight data. The exception is determined by scoring of flight data in relation to the flight triggers, the exception is declared when a cumulative predefined point threshold is exceeded during execution of the flight exercise. The method may include: detecting a voice command to initiate flight training and scoring passing off control from virtual instructor to user, detecting a voice command to pause flight training and scoring passing off control from user to virtual instructor, generate a halt context message when voice command occurred, provide the halt context message to the gai, and convert gai interaction text to audio output to provide voice feedback to the user. Halting the flight simulator further may include: generating an audio promptassociated with the determined exception, taking control of the simulated aircraft with the virtual instructor, and generating feedback context for a fine tuned model associated with qualities of the exception. The flight data is received via an application programming interface (API) associated with the flight simulator. The gai is trained with learning data associated with the received flight exercise selection. The flight data may include: control inputs, keystrokes, aircraft associated variables, weather, positional data references. The flight exercise defines flight parameters to be met in order to successfully complete the flight exercise. The flight parameters are associated with one or move triggers. The flight parameters are defined relative to a model predictive control system and a defined set of constraints. The flight simulator is one of Microsoft Flight Simulator™ (MSFS), X-plane™, and Prepar3d™ (P3D). A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method for providing a flight simulator virtual instructor training module interfacing with a home flight simulator. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0011] One general aspect includes a system for providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator a processor configured to execute instructions stored in a memory to perform: receiving a flight exercise selection from a user in the home flight simulator, monitoring flight triggers associated with the flight exercise maneuver, receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated with the flight exercise, determining that an exception has occurred when variables received in flight data exceed the flight triggers, determining a severity and style of the exception occurring and producing context feedback relevant to the severity, determining a score associated with the received flight data compared to ideal flight data relative to the flight triggers, pausing and resetting the flight simulator, commanding control of the flight simulator to return to a safe condition and demonstrate technique to the user, generating a context message identifying relevant variables exceeding flight triggers, providing the context message to a generative artificial intelligence (GAI), converting interaction textreceived from the gai to audio output, and delivering audio output to the user to provide verbal feedback on the exception to the user by the virtual instructor. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0012] One general aspect includes a method for providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator. The method also includes receiving a flight exercise selection from a user in the home flight simulator; monitoring flight triggers associated with the flight exercise maneuver, receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated the flight exercise, generating a context message identifying relevant variables exceeding flight triggers, providing the context message to a generative artificial intelligence (GAI), and converting interaction text received from the GAI to audio output. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0013] One general aspect includes a system for providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator a processor configured to execute instructions stored in a memory to perform: receiving a flight exercise selection from a user in the home flight simulator, monitoring flight triggers associated with the flight exercise maneuver, receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated the flight exercise, generating a context message identifying relevant variables exceeding flight triggers, providing the context message to a generative artificial intelligence (GAI), and converting interaction text received from the gai to audio output. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0014] Reference will now be made in detail to the embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.

[0015] FIG. 1 is a block diagram of an exemplary system 100 for providing a flight simulator virtual instructor training module interfacing with a home flight simulator, according to one embodiment of the disclosure. The system 100 comprises a home flight simulator software 120, a flight simulator virtual instructor training module 122, and a generative artificial intelligence (GAI) 124 / 150 such as for example a large language model (LLM) or generative pre-trained transformer (GPT). The GAI 124 may be executed on the computing device executing the flight simulator 120 or utilize a network based GAI 150, executed by one or more servers 140, containing one or more CPUs 142, coupled to non-volatile storage 144 with memory 146 executing instructions for providing the GAI 150.

[0016] The flight simulator system 100 is configured to provide a realistic and immersive flight simulation environment for a user, who may be a pilot-in-training, an experienced pilot or an enthusiast. The flight simulator system 100 may comprise a computer device, such as a desktop, laptop, tablet, or smartphone, and flight simulation software, such as Microsoft Flight Simulator™ (MSFS), X-Plane™, or Prepar3D™ (P3D). The home flight simulator system 100 may also comprise one or more computer processing units (CPU) 110, a memory 116 containing instructions for executing the flight simulator 120 and training module 122, display device 112, such as a monitor, a projector, or a virtual reality headset, and an input device, providing flight controls 120 such as a keyboard, a mouse, a joystick, a yoke, a flight control levers, a rudder, or a microphone. Non-volatile storage 114 is also provided to store programs and associated data. An audio input / output 118 interface enables the user to verbally interact with the training module 122. The processor 110 may be configured to execute instructions stored in the memory 116 to perform the method. The memory 116 may be configured to store data and instructions associated with the method, such as flight exercise selections, flight triggers, flight data, scores, context messages, interaction text, and audio output. A communication interface maybe configured to communicate with the home flight simulator 120 via the API 123, and with the GAI 150 via a network 190, such as the Internet, a local area network, or a wireless network. The flight simulator virtual instructor training module 122 may act as a virtual instructor that can monitor, evaluate, and instruct the user during the flight simulation, using natural language processing and machine learning techniques.

[0017] The home flight simulator 120 may allow the user to select and control a simulated aircraft, such as a fixed-wing or a rotary-wing aircraft, and to fly the simulated aircraft in various scenarios, such as different weather conditions, locations, terrains, airports, airspaces, and traffic. The home flight simulator 120 may also allow the user to fly the aircraft in a realistic way such as to practice known flight exercises, such as takeoff, landing, climb, descent, turn, stall, spin, emergency, or navigation. The home flight simulator 120 may generate and output flight data associated with the user performing the flight simulation, such as control inputs, keystrokes, aircraft associated variables, weather, positional data references, and user associated information. The home flight simulator 120 may communicate with the flight simulator virtual instructor training module 122 via an application programming interface (API) 123, which may allow the exchange of data and commands between the home flight simulator 120 and the flight simulator virtual instructor training module 122.

[0018] The flight simulator virtual instructor training module 122 is configured to implement a method for providing a flight simulator virtual instructor training module interfacing with the home flight simulator 110. The training module 122 comprises additional modules for providing the virtual flight simulator functionality such as but not limited to a model predictive control (MPC) manager 130, trigger manager 132 and sound manager 134. The exercise module 126 contains flight exercise that are prescribed to be performed to obtain a particular flight qualification against which parameters are defined that are to be met to qualify against. By selecting an exercise the exercise module monitors the home flight simulator 120 generated output for relevant criteria. The MPC 130 monitors flight data against the parameters in addition to audio data from sound manager 134 to determine flight status and when to communicate with the GAI 124 / 150 and control the flight simulator 120.

[0019] The GAI 124 / 150 is configured to receive the context message from the flight simulator virtual instructor training module 122, and to generate natural language interaction text based on the context message and a trained model. The GAI 124 / 150 may comprise a large language model (LLM), such as a generative pre-trained transformer (GPT) network, which is a type of deep neural network that can generate natural language text based on a given input and a large corpus of text data. The GAI 124 / 150 is trained with flight knowledge 180 from learning data associated with the flight exercise selection, such as aircraft specifications, aircraft control manuals, flight exercises, instructing best practices, common student confusion, and transportation agency flight test guides. The GAI 124 / 150 may use the learning data to fine tune the trained model and to generate natural language interaction text that is relevant, coherent, and informative for the user. The GAI 124 / 150 may communicate with the flight simulator virtual instructor training module 120 via the network 150.

[0020] A flight exercise maneuver may be for example a stall recovery maneuver, which may involve reducing the speed of the simulated aircraft until the angle of attack exceeds the critical angle of attack, resulting in a loss of lift and a stall. The stall recovery maneuver may require the user to apply full power, lower the nose, level the wings, and resume normal flight. The flight triggers may define the flight parameters to be met in order to successfully complete the stall recovery maneuver, such as speed, altitude, attitude, heading, pitch, roll, yaw, bank, angle of attack, lift, drag, thrust, or torque. The flight triggers may be associated with one or more thresholds, ranges, or limits that indicate the acceptable or optimal values or variations of the flight parameters. For example, the flight triggers may indicate that the speed should be between 40 and 60 knots, the altitude should be above 3000 feet, the attitude should be level, the heading should be constant, the pitch should be between -10 and 10 degrees, the roll should be between -10 and 10 degrees, the yaw should be zero, the bank should be zero, the angle of attack should be between 15 and 18 degrees, and the throttle position at 100%.

[0021] FIG. 2 is a system diagram of the flight simulator virtual instructor. The training module 122 performs various functions in the execution of the flight simulator virtual instructor. The MPC 129 interfaces with a trigger manager 132 and sound manager134 to provide the virtual instructor functionality. The trigger manager 132 process data references form the flight simulator 120 relative to defined flight parameters 212 associated with a particular exercise selected by the user. When a particular parameter is identified in the data references, for example altitude, a prediction is calculated by intention prediction 210 to identify what may occur based upon the current data references. A score can then be calculated by scoring module 214 to determine if the user has successfully performed the maneuver defined by the exercise. If the user makes an error or fails at the exercise control, control can be assumed by the MPC 129 to reset and demonstrate the correct maneuver. Access to the GAI can then be provided via a GAI control mechanism 216 which generates a context message for the GAI identifying the error that occurred and relevant flight data parameters. At the time of failure a cached audio message can be played identifying the type of failure that has occurred. The context message, along with the custom GAI 202 trained on data associated with flight instruction can then interact with the user by transforming text to speech 204 and speech to text 206 through audio output 222 and microphone input 220 interfaces. The sound manager 134 monitors the verbal interaction to identify commands to restart the flight simulator and provide control to the user.

[0022] The context message provided to the GAI may utilize a plurality of tags to aid in identifying relevant context information, for example a tag may be utilized to identify the experience level of the pilot and aircraft information to reduce the size of the context message provided to the GAI and provide consistency in messaging. The context message may also include messages related to flight characteristics such as for example “Tendency to Error in Heading” rather then providing specific heading values. A glossary may be utilized to map tags to specific values allowing for a reduction the size of the context message transmitted to the GAI.

[0023] FIG. 3 is a flowchart of an exemplary method 300 for providing a flight simulator virtual instructor training module interfacing with a home flight simulator, according to one embodiment of the disclosure. The method 300 may be performed by the flight simulator virtual instructor training module 122 in conjunction with the home flight simulator 120 and the GAI 124 / 150.

[0024] At step 302, the method may receive a flight exercise selection from the user in the home flight simulator 120. The flight exercise selection may indicate a flight exercise maneuver that the user wishes to perform and learn in the flight simulation, such as takeoff, landing, climb, descent, turn, stall, spin, emergency, or navigation. The flight exercise selection may be received via the input device of the home flight simulator 120, such as a keyboard, a mouse, a joystick, a yoke, a throttle, a rudder, or a microphone. The flight exercise selection may be communicated to the flight simulator virtual instructor training module 122 via the API 123.

[0025] At step 304, flight triggers associated with the flight exercise maneuver are monitored. The flight triggers may define flight parameters to be met in order to successfully complete the flight exercise maneuver, such as speed, altitude, attitude, heading, pitch, roll, yaw, bank, angle of attack, lift, drag, thrust, or torque. The flight triggers may be associated with one or more thresholds, ranges, or limits that indicate the acceptable or optimal values or variations of the flight parameters. The flight triggers may be retrieved from the memory 116 of the flight simulator virtual instructor training module 122, or from an external database or source. The flight triggers may be defined relative to a model predictive control system and a defined set of constraints, which may allow the flight simulator virtual instructor training module 122 to optimize the flight performance and safety.

[0026] Flight data is received at step 306 from the home flight simulator 120 associated with the user performing the maneuver flight simulation to meet criteria associated with the flight exercise. The flight data may comprise control inputs, keystrokes, aircraft associated variables, weather, positional data references, and user associated information. The flight data may be generated and output by the home flight simulator 120 based on the user's actions and responses in the flight simulation, and the simulated flight conditions and challenges. The flight data may be communicated to the flight simulator virtual instructor training module 122 via the API 123.

[0027] At step 308, it is determined if an exception has occurred when variables received in flight data exceed the retrieved flight triggers. The exception (Yes at 308)may indicate that the user has failed to meet the criteria associated with the flight exercise maneuver, or has performed the flight exercise maneuver in an incorrect, unsafe, or inefficient manner. The exception may be determined by comparing the variables received in flight data, such as speed, altitude, attitude, heading, pitch, roll, yaw, bank, angle of attack, lift, drag, thrust, or torque, with the retrieved flight triggers, such as thresholds, ranges, or limits, that indicate the acceptable or optimal values or variations of the flight parameters and predicting if a failure will occur. The exception may be declared when one or more variables received in flight data exceed the retrieved flight triggers, or when a cumulative predefined point threshold is exceeded during the execution of the flight exercise maneuver. The enforcement of thresholds may be handled in a way that responds to unsafe errors in a timely way as not to allow the user to deliver the aircraft to an unsafe scenario.

[0028] The severity and style of the exception occurring is determined at step 310. Context feedback can then be produced relevant to the severity.

[0029] A score is determined at step 312 that is associated with the received flight data compared to ideal flight data relative to the retrieved flight triggers. The score may indicate the degree of success or failure of the user in performing the flight exercise maneuver, or the degree of improvement or deterioration of the user's skills and knowledge. The score may be determined by comparing the received flight data with the ideal flight data, which may objectively access the errors of the user in a way that is consistent with tolerances expected by certified flight training. The score may be based on a scoring system that assigns points to the variables received in flight data according to their deviation from the ideal flight data.

[0030] At step 314, a pause and reset of the flight simulator 110 occurs. The pausing and resetting of the flight simulator 120 may occur when an exception is determined at step 312, or when a voice command from the user is detected, as described in connection with Fig. 4. The pausing and resetting of the flight simulator 110 may allow the flight simulator virtual instructor training module 122 to provide feedback and guidance to the user, and to prepare the flight simulator 120 for the next attempt or exercise. The pausing and resetting of the flight simulator 110 may be communicatedto the home flight simulator 120 via the API 123. The system at step 316 can then command control of the flight simulator to return to a safe condition and demonstrate technique to the user if required.

[0031] In some embodiments at step 315, the method 300 may further comprise taking control of the aircraft in the flight simulator 122 and demonstrating proper form to the user based on the flight exercise maneuver based upon the ideal flight data. The taking control of the aircraft in the flight simulator 120 may allow the flight simulator virtual instructor training module 122 to show the user how to perform the flight exercise maneuver correctly, safely, and efficiently, using the ideal flight data.

[0032] A context message is generated at step 318 identifying relevant variables exceeding flight triggers. The context message may comprise text data that describes the exception that occurred at step 308, and the relevant variables that exceeded the flight triggers, such as but not limited to speed, altitude, attitude, heading, pitch, roll, yaw, bank, angle of attack, lift, drag, thrust, or torque and the attempted exercise or maneuver. The context message may also comprise text data that describes the score that was determined at step 310, and the passing off control that was detected at step 316. The context message may be generated by the processor 110 of the flight simulator virtual instructor training module 122 based on the flight data, the flight triggers, the score, and the passing off control.

[0033] For example, the context message may indicate that the user performed the stall recovery maneuver with an exception, and the relevant variables that exceeded the flight triggers were: altitude = 2800 feet, roll = -15 degrees, yaw = 5 degrees, and bank = -15 degrees. The context message may also indicate that the user 140 scored 80 points out of 100 points, and that the user 140 passed off control to the virtual instructor by saying "You have control".

[0034] At step 320, the context message is provided to the GAI 124 / 150. The context message may be provided to the GAI 124 / 150 via the communication interface of the flight simulator virtual instructor training module 120 and the network 190. The context message may serve as an input for the GAI 124 / 150 to generate natural language interaction text based on the context message and the trained model.

[0035] I nteraction text received from the GA1 124 / 150 at step 322 is converted to audio output. The interaction text may comprise natural language text data that provides verbal feedback, guidance, and instruction to the user based on the context message and the trained model. The interaction text may be generated by the GAI 124 / 150 based on the context message and the trained model, and may be communicated to the flight simulator virtual instructor training module 122 via the network 190. The interaction text may be converted to audio output by the processor 110 of the flight simulator virtual instructor training module 122 using text-to-speech techniques.

[0036] At step 324, audio output is delivered to the user to provide verbal feedback on the exception to the user by the virtual instructor. The audio output may be delivered to the user via the audio device of the home flight simulator 110. The audio output may provide verbal feedback, guidance, and instruction to the user 140 based on the exception that occurred at step 308, the score that was determined at step 312, and the passing off control that was detected at step 316. The audio output may also provide verbal feedback, guidance, and instruction to the user based on the flight exercise maneuver and the ideal flight data. The user then can interact with the GAI at step 312 to help answer questions regarding the completion of the maneuver.

[0037] The interaction text may comprise natural language text data that provides verbal feedback, guidance, and instruction to the user based on the context message and the trained model. The interaction text may be generated by the GAI 124 / 150 based on the context message and the trained model, and may be communicated to the flight simulator virtual instructor training module 122 via the network 190. The interaction text may be converted to audio output by the processor 110 of the flight simulator virtual instructor training module 122 using text-to-speech techniques. The interaction text may provide the following verbal feedback, guidance, and instruction to the user "Good job on recovering from the stall, but you need to work on your altitude, roll, yaw, and bank. You lost 200 feet of altitude during the maneuver, which is too much. You also rolled and banked too much to the left, and yawed to the right, which caused you to lose directional control. You need to keep your wings level and your nose aligned with the horizon. You scored a 3 out of 4, which is not bad, but you can do better. I have taken control of the aircraft, and I will show you how to performthe stall recovery maneuver properly. Pay attention to the speed, altitude, attitude, heading, pitch, roll, yaw, bank, angle of attack, lift, drag, thrust, and torque. When you are ready, say 'I have control' and I will pass the control back to you."

[0038] FIG. 4 is a flowchart of an exemplary method 400 for interacting with a GAI by the flight training module 122. At step 402, the method 400 may detect a voice command to pause, or halt flight training and scoring passing off control between the user 140 and the virtual instructor. The voice command may be received via the microphone of the home flight simulator 120, and may indicate the user’s intention. The passing off control between the user and the virtual instructor may indicate the transfer of authority and responsibility for the simulated aircraft and the flight simulation between the user and the virtual instructor as would occur in real-world flight instruction. At step 404, the flight simulator 120 is halted as described at step 312. At step 406 a context message is generated and sent to the GAI at step 408. A response is received from the GAI at step 410 which is converted to audio at step 412. The audio may specify what the user did incorrectly or provide a query as to what further information the user may require regarding the exercise. The user may then interact with the GAI at step 414 with audio being converted at step 416 to text. If the user ‘assumes control ’at (Yes at 418) the simulator can be re-initiated at step 422. If the audio does not contain a command to assume control (No at 418) the interaction text is provided to the GAI at step 420.

[0039] FIG. 5 shows a method 500 for scoring an exercise according to one embodiment of the disclosure. The flight data may comprise control inputs, keystrokes, aircraft associated variables, weather, positional data references, and user associated information. The flight data may be generated and output by the home flight simulator 122 based on the user's actions and responses in the flight simulation, and the simulated flight conditions and challenges. The score may indicate the degree of success or failure of the user in performing the flight exercise maneuver, or the degree of improvement or deterioration of the user's skills and knowledge. The score may be determined by comparing the received flight data with the ideal flight data, which may indicate the optimal or desired values or variations of the flight parameters for the flight exercise maneuver. The score may be based on a scoringsystem that assigns points to the variables received in flight data according to their deviation from the ideal flight data. For example, the flight data may indicate that the user performed the stall recovery maneuver with the following values: speed = 50 knots, altitude = 2800 feet, attitude = 5 degrees nose down, heading = 180 degrees, pitch = -5 degrees, roll = -15 degrees, yaw = 5 degrees, bank = -15 degrees, angle of attack = 17 degrees, lift = 0.9 times the weight, drag = 0.3 times the weight, thrust = 1.1 times the weight, and torque = 10 percent. The flight data may be arrays of datapoints over the maneuver observation period. The score system may use the same strike method a human flight examiner would take where errors are recorded and reduce the score of the flight exercise, based on a defined scoring system associated with the parameters of the selected exercise errors are recorded and a score is given.

[0040] At step 502 a flight selection is received which contains the parameters of the exercise to be performed. The flight data is received at step 504 and compared to the parameters associated with the exercise. If the parameters are outside a defined threshold, for example speed < 50 knots or a cumulative score is below a defined value, an exception can be declared at step 506. The exception can be used to calculate the score at step 508 or be cumulatively calculated. When the threshold is exceeded (Yes at step 510) an predefined audio file can be played back at step 512 prior to initiating the context message to the GAI module 124 / 150. The exception threshold category may determine the severity of the cached audio file and GAI module 124 / 150 context.

[0041] The foregoing description of the embodiments of the present disclosure has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the present disclosure to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the present disclosure be limited not by this detailed description, but rather by the claims of this application. As will be understood by those familiar with the art, the present disclosure may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Likewise, the particular naming and division of the modules, routines, features, attributes, methodologies andother aspects are not mandatory or significant, and the mechanisms that implement the present disclosure or its features may have different names, divisions and / or formats. Furthermore, as will be apparent to one of ordinary skill in the relevant art, the modules, routines, features, attributes, methodologies and other aspects of the present disclosure can be implemented as software, hardware, firmware or any combination of the three. Of course, wherever a component of the present disclosure is implemented as software, the component can be implemented as a standalone program, as part of a larger program, as a plurality of separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and / or in every and any other way known now or in the future to those of skill in the art of computer programming. Additionally, the present disclosure is in no way limited to implementation in any specific operating system or environment. Accordingly, the disclosure of the present disclosure is intended to be illustrative, but not limiting, of the scope of the present disclosure, which is set forth in the following claims.

Claims

CLAIMS:

1. A computer-implemented method of providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator executed on a computing device, the method comprising: receiving a flight exercise selection from a user in the home flight simulator; monitoring flight triggers associated with the flight exercise maneuver; receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated the flight exercise; determining that an exception has occurred when variables received in flight data exceed the flight triggers; determining severity and style of the exception occurring and producing context feedback relevant to the severity; determining a score associated with the received flight data compared to ideal flight data relative to the flight triggers; pausing and resetting the flight simulator; commanding control of the flight simulator to return to a safe condition and demonstrate technique to the user; generating a context message identifying relevant variables exceeding flight triggers; providing the context message to a generative artificial intelligence (GAI); converting interaction text received from the GAI to audio output; and delivering audio output to the user to provide verbal feedback on the exception to the user by the virtual instructor.

2. The method of claim 1 wherein subsequent to pausing and resetting the flight simulator the method further comprising:taking control of the aircraft in the flight simulator and demonstrating proper form to the user based on the flight exercise maneuver based upon the ideal flight data.

3. The method of claim 1 wherein the exception is determined by scoring of flight data in relation to the flight triggers, the exception is declared when a cumulative predefined point threshold is exceeded during execution of the flight exercise.

4. The method of claim 1 further comprising: detecting a voice command to initiate flight training and scoring passing off control from virtual instructor to user; detecting a voice command to pause flight training and scoring passing off control from user to virtual instructor; generate a halt context message when voice command occurred; provide the halt context message to the GAI; and convert GAI interaction text to audio output to provide voice feedback to the user.

5. The method of claim 1 wherein halting the flight simulator further comprises: generating an audio prompt associated with the determined exception; taking control of the simulated aircraft with the virtual instructor; and generating feedback context for a fine tuned model associated with qualities of the exception.

6. The method of claim 1 wherein the flight data is received via an application programming interface (API) associated with the flight simulator.

7. The method of claim 1 wherein the GAI is trained with learning data associated with the received flight exercise selection.

8. The method of claim 7 wherein the learning data is selected from a group comprising: aircraft specifications, aircraft control manuals, flight exercises; instructing best practices, common student confusion and transportation agency flight test guides.

9. The method of claim 1 wherein the flight data comprises: control inputs, keystrokes, aircraft associated variables, weather, positional data references, and user associated information.

10. The method of claim 1 wherein the flight exercise defines flight parameters to be met in order to successfully complete the flight exercise.11 . The method of claim 10 wherein the flight parameters are associated with one or move triggers.

12. The method of claim 10 wherein the flight parameters are defined relative to a model predictive control system and a defined set of constraints.

13. The method of claim 1 wherein the flight simulator is one of Microsoft Flight Simulator (MSFS), X-Plane, and Prepar3D (P3D).

14. A system for providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator, the system comprising: a processor configured to execute instructions stored in a memory to perform: receiving a flight exercise selection from a user in the home flight simulator; monitoring flight triggers associated with the flight exercise maneuver; receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated with the flight exercise; determining that an exception has occurred when variables received in flight data exceed the flight triggers;determining a severity and style of the exception occurring and producing context feedback relevant to the severity; determining a score associated with the received flight data compared to ideal flight data relative to the flight triggers; pausing and resetting the flight simulator; commanding control of the flight simulator to return to a safe condition and demonstrate technique to the user; generating a context message identifying relevant variables exceeding flight triggers; providing the context message to a generative artificial intelligence (gai); converting interaction text received from the gai to audio output; and delivering audio output to the user to provide verbal feedback on the exception to the user by the virtual instructor.

15. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method for providing a flight simulator virtual instructor training module interfacing with a home flight simulator, the method according to any one of claims 1 to 13.

16. A method for providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator, the method comprising: receiving a flight exercise selection from a user in the home flight simulator; monitoring flight triggers associated with the flight exercise maneuver; receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated the flight exercise; generating a context message identifying relevant variables exceeding flight triggers; providing the context message to a generative artificial intelligence (GAI); andconverting interaction text received from the GAI to audio output.

17. A system for providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator, the system comprising: a processor configured to execute instructions stored in a memory to perform: receiving a flight exercise selection from a user in the home flight simulator; monitoring flight triggers associated with the flight exercise maneuver; receiving flight data from the home flight simulator associated with the user performing the flight exercise maneuver in the flight simulator to meet criteria associated the flight exercise; generating a context message identifying relevant variables exceeding flight triggers; providing the context message to a generative artificial intelligence (GAI); and converting interaction text received from the GAI to audio output.

18. A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform a method for providing an aircraft flight simulator virtual instructor training module interfacing with a home flight simulator, the method according to any one of claim 15.

Images