A method and system for pilot and flight dispatcher collaborative training

Abstract

The application discloses a kind of pilot and flight dispatching officer cooperative training method and system, it is related to aviation flight training field, by constructing the cooperative training environment of virtual and real combination, integrate scene generation and management, multi-modal data acquisition, data fusion processing, performance and competency evaluation, individualized feedback and other functional modules, realize the collaborative operation and ability evaluation of pilot and dispatching officer in simulation training scene.System acquisition includes multidimensional data such as eye movement, physiology, voice, operation behavior, use the deep learning model based on attention mechanism for feature fusion, and combine competency model to quantitatively evaluate the key ability such as procedure application, communication, leadership and team cooperation in training process.Effectively solve the problem that pilot and dispatching officer training is isolated, lack of cooperative drill and data-driven evaluation in existing training system, significantly improve the collaborative decision-making and disposal ability under complex special situation, have higher practicality and popularization value.

Description

Technical Field

[0001] This invention relates to the field of aviation flight training technology, and in particular to a method and system for collaborative training of pilots and flight dispatchers. Background Technology

[0002] In the field of aviation flight training and decision support, existing simulation training systems mostly focus on the skills training of one party, such as providing flight simulators for pilots or dispatch simulation systems for dispatchers. There is a lack of an integrated environment capable of fusing multi-dimensional information from both parties and conducting joint decision-making exercises. Furthermore, the massive amounts of interactive data, physiological data, and environmental data generated during training have not been systematically collected, integrated, and analyzed, thus hindering in-depth optimization and personalized feedback of training effectiveness based on data-driven approaches.

[0003] In modern aviation operations, flight safety and efficiency heavily rely on efficient collaboration and informed decision-making between pilots (in the air) and dispatchers (on the ground). However, in traditional training models, pilot and dispatcher training is often independent, lacking collaborative training in realistic emergency scenarios. This can lead to risks in actual operations due to poor communication, misunderstandings, or inconsistent decision-making processes.

[0004] Therefore, a method and system for collaborative training between pilots and flight dispatchers is provided to solve the above problems. Summary of the Invention

[0005] To address the aforementioned challenges, this invention provides a method and system for collaborative training of pilots and flight dispatchers. By constructing a virtual-real collaborative training environment that integrates various scenario data and multimodal physiological data, it generates personalized feedback, training reports, and improvement suggestions. This addresses the shortcomings of existing training in two key areas: pilot-dispatcher collaborative handling of realistic emergencies and personalized assessment of the collaborative effectiveness of the two roles. It enhances the collaborative work, decision-making, and handling capabilities of pilots and dispatchers under complex emergencies, enabling refined and quantitative assessment and improvement of collaborative capabilities, and effectively enhancing aviation safety.

[0006] To achieve the above objectives, the present invention provides a method and system for collaborative training between pilots and flight dispatchers, comprising the following steps: S1: Construct simulated training scenarios that include daily operations or special event scenarios; S2: Based on the simulated training scenario, set training tasks for pilots and flight dispatchers respectively; S3: Establish a multimodal perception training environment, which includes a pilot simulation terminal, a flight dispatcher simulation terminal, and a multimodal data acquisition system for collecting data; the multimodal data includes physiological data, operational behavior data, and voice data of the pilot and flight dispatcher; S4: Pilots and dispatchers collaborate on the same simulated training scenario and collect multimodal data during the operation. S5: The collected operational behavior data and voice data are fused and processed to generate a collaborative decision-making process data stream; the fusion processing includes data reception, time synchronization, feature extraction, and feature-level fusion; feature-level fusion is achieved using a deep learning model based on an attention mechanism; S6: The collected physiological indicators of pilots and dispatchers are fused separately to generate their respective physiological characterization data streams; S7: Based on the collaborative decision-making process data flow, a preset performance evaluation model is used to comprehensively evaluate the collaborative performance of pilots and dispatchers, and obtain the performance evaluation results; S8: Based on the collaborative decision-making process data stream, the physiological representation data stream, and the performance evaluation results, the competency indicators and observable behaviors of pilots and dispatchers are analyzed and evaluated using a preset competency evaluation model to obtain competency evaluation results. S9: Generate personalized feedback and training reports based on performance evaluation results and competency assessment results.

[0007] Preferably, the simulated training scenario is generated by an artificial intelligence model after performing feature analysis on historical operation and training data.

[0008] Preferably, the multimodal data acquisition system is used to acquire: Eye movement data, physiological data, voice data, and operational behavior data from the pilot simulator; Operation log data, voice data, and screen interaction data from the dispatcher's simulation terminal; Aircraft state parameters and scenario event data in simulated flight environments.

[0009] Preferably, the deep learning model based on the attention mechanism is the Transformer model, which aligns the pilot's eye-tracking heatmap with the dispatcher's interface operation feature sequence to generate a decision-response correlation graph and identify decision-response delays or information gaps between the pilot and the dispatcher.

[0010] Preferably, the quantitative evaluation indicators of the performance evaluation model include decision-making timeliness, operational accuracy, communication effectiveness, and workload. The decision-making timeliness is quantified by one or more indicators among fault identification time, initial solution proposal time, and final decision execution time. The communication effectiveness is quantified by one or more indicators among standard language usage rate, communication repetition completeness rate, and information request-response delay.

[0011] Preferably, the performance evaluation model adopts the fuzzy comprehensive evaluation method, combined with the AHP (Analytic Hierarchy Process) to determine the weight of each dimension and output a comprehensive performance score.

[0012] Preferably, it also includes: dynamically adjusting the difficulty or task settings of subsequent simulation training scenarios based on performance evaluation results and competency evaluation results to achieve adaptive training.

[0013] A pilot-dispatcher collaborative training system includes: The scenario generation and management module is used to build, store, and distribute simulation training scenarios; The multimodal data acquisition module is used to collect data from the pilot's terminal, the dispatcher's terminal, and the simulated environment; The data fusion and processing module is used to fuse the collected operational behavior data and voice data to obtain a fused collaborative process data stream, and to fuse the collected physiological indicators of pilots and dispatchers to obtain a fused physiological characterization data stream. The performance evaluation module is used to comprehensively evaluate the collaboration performance based on the fused collaboration process data flow and using a performance evaluation model to obtain the performance evaluation result. The competency assessment module is used to analyze and evaluate competency indicators and observable behaviors based on the fused collaborative process data stream and the fused physiological representation data stream, using a competency assessment model to obtain competency assessment results. The report generation module is used to generate personalized feedback and training reports based on performance evaluation results and competency assessment results.

[0014] Preferably, it also includes: an adaptive training control module, used to dynamically adjust the difficulty or task settings of subsequent simulated training scenarios based on performance evaluation results and competency evaluation results, so as to achieve adaptive training.

[0015] Therefore, the present invention, employing the aforementioned method and system for collaborative training of pilots and flight dispatchers, has the following beneficial effects: (1) This invention breaks the training silo: it enables pilots and dispatchers to conduct collaborative training in a highly simulated joint environment, effectively simulating the collaborative decision-making process in real work.

[0016] (2) This invention introduces multimodal perception: by collecting multi-dimensional information such as biological data, behavioral data, and environmental data, it is possible to more comprehensively and objectively assess the cognitive load, decision-making status, and collaborative efficiency of trainees.

[0017] (3) This invention realizes quantitative evaluation and intelligent feedback: it uses a data-driven approach to conduct a refined evaluation of the effectiveness of collaborative decision-making and provides visualized and personalized feedback, which significantly improves the scientificity and efficiency of training.

[0018] (4) This invention improves the scenario construction method, realizing the transformation from manual input scenario design to data analysis based on production needs and the use of artificial intelligence models to process and optimize scenario design.

[0019] (5) The present invention has adaptive capability: the training approach can be dynamically adjusted according to the trainee's level, and the training is always challenging and effective.

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of a pilot-flight dispatcher collaborative training method in this invention; Figure 2 This is a schematic diagram of the scenario generation and management process in an embodiment of the present invention; Detailed Implementation

[0022] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0024] The terms "comprising" or "including" as used in this invention mean that the element preceding the term encompasses the element listed after the term, but do not exclude the possibility of encompassing other elements. Those skilled in the art can understand the specific meaning of these terms in this invention according to the specific circumstances.

[0025] Example 1 A collaborative training method for pilots and flight dispatchers, such as Figure 1 As shown, it includes the following steps: S1: Construct simulated training scenarios that include daily operations or special event scenarios; Specifically, based on historical flight data and expert experience, simulation training scenarios suitable for collaboration between pilots and flight dispatchers are constructed through human or artificial intelligence input or arrangement of data for one or more scenarios; special situation training scenarios include one or more of engine failure, severe weather, and system failure.

[0026] S2: Based on the simulated training scenario, set training tasks for pilots and flight dispatchers respectively; Specifically, based on simulated training scenarios, pilots and flight dispatchers are assigned one or more tasks manually or using artificial intelligence, and the software and hardware used by the flight trainer are compatible (capable of inputting scenarios and tasks into the trainer, and the trainer can output training data to this platform). The simulated training scenarios are generated by the artificial intelligence model after performing feature analysis on historical operation and training data.

[0027] S3: Establish a multimodal perception training environment, which includes a pilot simulation terminal, a flight dispatcher simulation terminal, and a multimodal data acquisition system for collecting data; the multimodal data includes physiological data, operational behavior data, and voice data of the pilot and flight dispatcher; The pilot simulator contains eye-tracking data, physiological data, voice data, and operational behavior data. Eye-tracking data includes fixation point and pupil diameter; physiological data includes heart rate and skin conductance; voice data includes communication content, tone, and speech rate; and operational behavior data includes control inputs and switching actions. The instruments used can be eye trackers, microphones, sensors, and cameras. The dispatcher's simulation terminal's operation log data, voice data, and screen interaction data are collected through screen recording, operation logs, and microphone. The operation logs include click streams and query actions, the voice data is voice communication data, and the screen interaction data is screen recording data. The simulated flight environment includes aircraft status parameters (weather, aircraft parameters, airspace information) and scenario event data. Flight status data includes attitude, speed, and heading, scenario events include fault injection and weather changes, and spatiotemporal information includes timestamps and geographical locations.

[0028] S4: Pilots and dispatchers collaborate on the same simulated training scenario and collect multimodal data during the operation. S5: The collected operational behavior data and voice data are fused and processed to generate a collaborative decision-making process data stream; the fusion processing includes data reception, time synchronization, feature extraction, and feature-level fusion; feature-level fusion is achieved using a deep learning model based on an attention mechanism; The deep learning model based on the attention mechanism is the Transformer model, which aligns the pilot's eye-tracking heatmap with the dispatcher's interface operation feature sequence to generate a decision-response correlation graph, and identifies decision-response delays or information gaps between the pilot and the dispatcher.

[0029] Specifically, the system receives and synchronizes multi-source heterogeneous data from a multimodal data acquisition system in real time. Data preprocessing and cleaning, primarily involving filtering, noise reduction, and formatting, are performed. Feature extraction and semantic understanding techniques are used to fuse the data, generating a fused collaborative decision-making process data stream. An attention-based deep learning model or traditional multi-source information fusion algorithms are employed to achieve efficient integration and semantic understanding of the heterogeneous data. This embodiment utilizes a Transformer-based multimodal fusion model to align the pilot's eye-tracking heatmap with the dispatcher's screen operation sequence, generating a "decision-response" correlation graph. This identifies the collaborative gap between the pilot's failure to promptly check instruments after engine failure and the dispatcher's failure to provide timely alternate landing information. The output includes a structured collaborative decision-making process data stream, key event segment markers, and visualized intermediate results.

[0030] S6: The collected physiological indicators of pilots and dispatchers are fused separately to generate their respective physiological characterization data streams; the process is performed as described in S5.

[0031] S7: Based on the collaborative decision-making process data flow, a preset performance evaluation model is used to comprehensively evaluate the collaborative performance of pilots and dispatchers, and obtain the performance evaluation results; The quantitative evaluation indicators of the performance evaluation model include decision-making timeliness, operational accuracy, communication effectiveness, and workload. Decision-making timeliness is quantified by one or more indicators among fault identification time, initial solution proposal time, and final decision execution time. Communication effectiveness is quantified by one or more indicators among standard language usage rate, communication repetition completeness rate, and information request-response delay.

[0032] Specifically, based on the fused data stream, a pre-defined evaluation model is used to conduct real-time or post-event comprehensive evaluations of the collaborative decision-making process and results between pilots and dispatchers. The evaluation model includes a quantitative evaluation index system encompassing multiple dimensions such as decision-making timeliness, operational accuracy, communication effectiveness, and workload level. Decision-making timeliness is quantified using one or more indicators among fault identification time, initial solution proposal time, and final decision execution time. Operational accuracy is quantified using one or more indicators among procedure execution compliance rate, decision-manual conformity, and parameter calculation accuracy. Communication effectiveness is quantified using one or more indicators among standard terminology usage rate, communication repetition completeness rate, information request-response delay, and redundant information ratio. Workload level is quantified using one or more indicators among operational sequence efficiency, multi-task switching frequency, heart rate variability (HRV), and the rationality of task priority allocation. A fuzzy comprehensive evaluation method is used, combined with the Analytic Hierarchy Process (AHP) to determine the weights of each dimension, outputting a comprehensive performance score. For example, in this training, the "communication effectiveness" dimension scored low due to repeated failures to use standard terminology; the system generated targeted training suggestions accordingly.

[0033] S8: Based on the collaborative decision-making process data flow, the physiological representation data flow, and the performance evaluation results, the competency indicators and observable behaviors of pilots and dispatchers are analyzed and evaluated using a preset competency evaluation model to obtain competency evaluation results; the competency evaluation model includes: competency and observable behavior evaluation indicators similar to those of pilots and dispatchers, and competency and observable behavior evaluation indicators with different names matched to pilots and dispatchers.

[0034] The core of this step lies in combining data-driven performance evaluation with competency models from behavioral science to achieve deep insights from "what was done" to "how well it was done" and "why it was done this way." Specifically, this includes: The competency dimension mapping first involves mapping the quantitative indicators output by the performance evaluation model (such as decision-making timeliness, operational accuracy, and communication effectiveness) to preset competency dimensions (eight competencies each for pilots and dispatchers, as shown in Table 1 below). These dimensions include at least procedural application, communication, situational awareness, problem-solving and decision-making, leadership and teamwork, and workload management.

[0035] Table 1. Competency Table for Pilots and Dispatchers

[0036] Next, multimodal data correlation analysis is performed. The system deeply correlates and analyzes the pilot's physiological representation data stream (such as stress levels reflected by changes in heart rate and skin conductance, and attention allocation reflected by eye-tracking heatmaps) with the dispatcher's screen interaction data stream. For example, when an "engine failure" occurs in the simulated scenario, the system uses deep learning models such as Transformer to align the pilot's eye-tracking sequence at critical decision moments (whether key instruments were scanned in time) with the alternate landing information provided by the dispatcher on the timeline, generating a decision-response correlation graph. This aims to identify specific observable behaviors, such as "the pilot did not check the N1 tachometer within 3 seconds after the engine warning" or "the dispatcher's response delay after receiving the pilot's alternate landing request exceeds a specified threshold."

[0037] Competency decoding, based on the aforementioned correlation analysis, utilizes a pre-defined competency assessment model to decode and infer the knowledge, skills, and attitudes behind observable behaviors, as well as the motivation, ability, and situational cues driving those behaviors—namely, KSAs and MAP models. For example, the behavior of "failing to check the instruments in a timely manner" is associated with a weakness in the "situational awareness" dimension, and may be further inferred to be caused by excessive workload (supported by heart rate variability and operation frequency) or unfamiliarity with standard procedure knowledge.

[0038] Finally, a comprehensive quantitative assessment is conducted. The system employs fuzzy comprehensive evaluation and combines it with the Analytic Hierarchy Process (AHP) to assign weights to each competency dimension and subordinate behavioral indicators. By integrating performance data and physiological behavioral data, the system calculates the quantitative scores and overall ratings for each trainee across all competency dimensions. This completes the transformation and evaluation from low-level operational data to high-level competency indicators, providing a direct basis for generating in-depth personalized feedback reports.

[0039] The assessment report utilizes a deep learning model to analyze processed multimodal physiological data and comprehensive data processed by a performance evaluation model. It understands and learns the relevance and collaborative characteristics of competencies, including the knowledge, skills, attitudes, observable behaviors, motivations, abilities, and prompts required for these behaviors. This generates a visual assessment report on competencies, including but not limited to collaborative performance, competency ratings, and the effectiveness of competency collaboration (building a competency collaboration model based on user application of procedures, communication, situational awareness, leadership and teamwork, and workload management).

[0040] S9: Generate personalized feedback and training reports based on performance evaluation results and competency assessment results.

[0041] Based on the evaluation results, a visual analysis report and personalized improvement suggestions are automatically generated, and trainees are provided with key decision point replays, expert example comparisons, and targeted reinforcement training modules.

[0042] This embodiment can also dynamically adjust the difficulty coefficient of subsequent training based on the evaluation results. For example, it can suddenly inject new faults or severe weather conditions to achieve adaptive training.

[0043] Example 2 A pilot-dispatcher collaborative training system includes: The scenario generation and management module is used to build, store, and distribute simulation training scenarios; The multimodal data acquisition module is used to collect data from the pilot's terminal, the dispatcher's terminal, and the simulated environment; The data fusion and processing module is used to fuse the collected operational behavior data and voice data to obtain a fused collaborative process data stream, and to fuse the collected physiological indicators of pilots and dispatchers to obtain a fused physiological characterization data stream. The performance evaluation module is used to comprehensively evaluate the collaboration performance based on the fused collaboration process data flow and using a performance evaluation model to obtain the performance evaluation result. The competency assessment module is used to analyze and evaluate competency indicators and observable behaviors based on the fused collaborative process data stream and the fused physiological representation data stream, using a competency assessment model to obtain competency assessment results. The report generation module is used to generate personalized feedback and training reports based on performance evaluation results and competency assessment results.

[0044] The adaptive training control module is used to dynamically adjust the difficulty or task settings of subsequent simulation training scenarios based on performance evaluation results and competency evaluation results, so as to achieve adaptive training.

[0045] Example 3 This example demonstrates the process of generating and managing a scenario where "pilots and dispatchers lack sufficient understanding of takeoff regulations." The process overview is as follows: Figure 2 As shown, the details are as follows: On a certain day, during flight operations, an airline pilot, under IFR weather conditions, received an instrument flight release from the dispatcher. After seeing other airline pilots abort their takeoffs, the pilot asked the dispatcher whether to proceed. The dispatcher told the pilot to decide for themselves. After taxiing a distance on the taxiway, the weather conditions changed. The pilot did not hear the air traffic controller's report on the weather conditions when starting to taxi, and the dispatcher did not update the weather conditions in time. At an altitude of 300 feet, the aircraft encountered low-altitude wind shear and poor visibility. The captain quickly took control of the aircraft, pulled back on the stick, and turned to avoid the obstacle, constituting a serious incident.

[0046] Based on this incident, it was determined that both the pilots and dispatchers lacked sufficient understanding of relevant regulations during takeoff, and their competence in "procedure application" was weak. Therefore, it was necessary to develop collaborative training courses. The airline input relevant vocabulary, regulations, flight data, accident report information, and other materials into the system of this embodiment. The system processed the relevant data and materials as follows: S1: Build a vocabulary database (flight and dispatch terminology, as well as professional terminology and corresponding programming codes), build a data database (accident and accident symptom compilations, regulations, Quick Reference Handbook, etc., and their corresponding programming codes), and build a database (QAR, NSR, FOQA, etc.). S2: Data denoising, data filtering, and data fusion processing (matching with suitable data) to generate an optimized database; S3: Extract effective data information from the optimized database, combined with manually input questions; S4: Edit scenario scripts and task scripts; S5: Based on the scenario script content, a unified flight training language specification is constructed by referencing a vocabulary database. S6: Construct simulated training scenarios containing tasks; S7: Archive scenarios and their tasks in a scenario library, and build a scenario and task design meta-model based on the scenario library.

[0047] The specific case involved in this case is as follows: S1: Building and Integrating Resource Repositories Database access: Accesses the definitions of dispatcher responsibilities in the "General Operations Manual", "Flight Crew Operations Manual", and "Flight Dispatcher Training Syllabus".

[0048] Database access: Retrieve underlying data from recent QAR data regarding events such as "takeoff configuration warning" and "V1 aborted takeoff"; analyze observation records in LOSA reports regarding "effectiveness of pre-takeoff briefing".

[0049] Accident Symptoms Compilation: In-depth analysis of the case you provided, "Fuzzy decision-making under IFR conditions leads to low-altitude wind shear".

[0050] S2: Data Fusion and Problem Extraction The manual input question is: "Verify whether there are systemic weaknesses in the implementation of regulations and procedures by pilots and dispatchers during the takeoff phase, especially in the division of decision-making responsibilities and information communication."

[0051] The system extracted valid information: Regulations clearly stipulate that "flight dispatchers and captains are jointly responsible for release," but in the extracted cases and data, this responsibility was blurred in dynamic decision-making. The dispatcher did not provide decision support, and the pilot did not request a reassessment, indicating that both parties' understanding of the procedural implications of "joint release" remained at the level of static documents rather than a dynamic process.

[0052] Communication: Data shows that information flow was disrupted at critical junctures of changing weather conditions. Dispatchers failed to proactively update information, and pilots failed to repeat key instructions, resulting in a failure of the "request-provide" two-way communication model.

[0053] Leadership and Team Collaboration: The case study lacked a clear "team leader" to integrate resources and make final decisions. The dispatcher relinquished his leadership and support role, and the pilot made decisions in isolation, reflecting the breakdown of the team collaboration structure under pressure.

[0054] Analysis conclusion: The airline's conjecture is correct but incomplete. The weakness in "procedure application" is a superficial phenomenon; the root cause lies in poor "communication" and the misplacement of roles in "leadership and teamwork." These three competencies are interconnected, forming a "decision-making collaboration chain." The breakage of any link in this chain will lead to overall failure. Therefore, training must focus on the synergy of these three competencies.

[0055] After drawing conclusions from the analysis, the system designed two simulation training scenario scripts: "Right engine failure after takeoff" and "Takeoff role under complex weather conditions".

[0056] Example 4 This example demonstrates a collaborative training process for a scenario where "one engine fails after flight takeoff," as detailed below: (1) Idea loading: The training system loads the idea of ​​"single engine failure after takeoff", initializes the simulation environment, including: the airport is an international airport, initial weight, weather conditions (crosswind 5 knots), and synchronizes the idea information to the pilot simulation cockpit and dispatcher workstation.

[0057] (2) Environment setup and data acquisition: The pilot enters the high-fidelity simulated cockpit, and the dispatcher logs into the dispatch simulation system. The multimodal acquisition system is activated, including: the in-cockpit camera begins to track the pilot's eye movements, the microphone records in-cockpit communications and the pilot's voice, the manipulation sensor records the operation input, and the heart rate wristband monitors physiological signals; the screen recording software on the dispatcher's end records the operation process, and the voice communication system records the dialogue between the dispatcher and the pilot.

[0058] (3) Collaborative Training Execution: The thought process triggers an "engine failure" malfunction. The pilot immediately executes the memorized items (such as "V1 cutoff" or "continue takeoff" decisions) and reports the situation to the dispatcher via radio. Upon receiving the information, the dispatcher quickly invokes system tools, such as the performance calculation module, alternate landing site selection module, and route weather query module, to assess the possibility of returning to the airport or making an alternate landing, calculate the optimal landing weight, and provide decision support suggestions to the pilot (such as "recommend returning to the airport for landing, the runway is clear"). During this process, both parties engage in multiple communications and collaborative decision-making processes.

[0059] (4) Data fusion and processing: The system backend receives and aligns all data streams in real time. For example, the timing of the pilot’s “Mayday” call is aligned and fused with the content displayed on the dispatcher’s screen at that moment (such as the alternate landing information being queried) to form a complete decision context semantic information.

[0060] (5) Performance evaluation: After training, evaluate the model operation. For example, evaluate the decision time of the pilot from the occurrence of the fault to the first declaration (timeliness), whether the alternate landing site selection provided by the dispatcher meets the company policy and performance requirements (accuracy), whether both parties used standard terminology and recited it completely in the communication (communication effectiveness), and the pilot's heart rate variability in emergency situations (load level), etc.

[0061] (6) Feedback and Reporting: The system generates a report indicating that the dispatcher's operation of querying alternate airport weather information was delayed by 15 seconds in this scenario. The system then recommends that the dispatcher conduct a special training module on "rapid information retrieval" and plays a video of a decision-making expert handling the situation under the same scenario as an example for the pilot.

[0062] Example 5 This example demonstrates a collaborative analysis report based on the concept of "one engine failure after flight takeoff" after collaborative training, as detailed below: Personalized training improvement suggestion report: Scenario ID: GEN-ENG-FAIL-001; Scenario Name: Right engine fails after takeoff; Training team: Pilots [XX1] | Dispatchers [XX2]; Training date: [Date]; Overall performance summary: The crew successfully completed a single-engine return and landed safely, demonstrating basic scenario handling capabilities. However, during the critical initial decision-making phase, team collaboration was delayed, and communication efficiency had significant room for improvement, impacting the overall smoothness and safety of the operation.

[0063] Suggestions for improvement in specific areas: 1. Program Application - Rating: 3.5 / 5.0; Advantages: Pilots execute engine failure memory programs accurately, and dispatchers can quickly initiate emergency checklists.

[0064] Areas for improvement: Decision delay: From engine failure to the final determination of an alternate landing site, the time taken was 45 seconds longer than the optimal baseline. Analysis revealed that both parties hesitated when jointly deciding on an alternate landing site, and the dispatcher initially provided too much unfiltered alternate landing site information, increasing the workload of the pilots.

[0065] Suggestions for improvement: Pilots: Strengthen training to improve their ability to quickly process information and make decisions in emergency situations.

[0066] Dispatcher: Practice using the "decision matrix" method to prioritize alternate airports in advance based on distance, weather, and runway length, and provide the crew with a "preferred suggestion" and a brief reason.

[0067] 2. Communication - Rating: 3.0 / 5.0; Advantages: Initial contact was established and key information was communicated.

[0068] Areas for improvement: Lack of standard terminology: When pilots first contacted the dispatcher, they did not use the standard term "declare an emergency," resulting in reduced initial communication efficiency.

[0069] Duplicate information request: Due to incomplete initial information transmission, the dispatcher had to perform a second information confirmation, wasting valuable time.

[0070] Suggestions for improvement: Both parties: Conduct specialized training on standard communication phrases for emergency situations, and be sure to follow the standardized information flow of "emergency type - intent - number of passengers - remaining fuel".

[0071] We adhere to the principle of "first-time accuracy": in the initial communication, we strive to convey all core information completely and accurately.

[0072] 3. Leadership and Teamwork - Rating: 3.2 / 5.0; Advantages: Clear division of roles during the emergency landing phase.

[0073] Areas for improvement: Role ambiguity: In the early stages of the incident, there was a lack of a clear "team leader" to drive the decision-making process. Pilots were waiting for information, while dispatchers were passively providing data, creating a "decision vacuum."

[0074] Physiological data corroborate this: the pilot's skin conductance remained elevated during the decision delay, indicating that he felt decision-making pressure but did not effectively translate it into leadership behavior.

[0075] Suggestions for improvement: Clearly define leadership roles: Training should emphasize that in emergency situations, the pilot is the final decision-maker, while the dispatcher is the decision support manager. Both parties need to proactively assume their respective leadership responsibilities.

[0076] Training in proactive leadership statements: The pilot should use: "I now need two alternative landing options"; The dispatcher should use: "I recommend diverting to... for the reason of..., do you agree?" Recommended Targeted Enhancement Simulation Training Scenarios: Scenario A-EP-01: "High-Intensity Team Decision-Making Simulation under Single-Failure Conditions"; Scenario C-COM-03: "Standardized Terminology for Emergency Communication and Initial Information Transmission Drill"; Scenario T-LDR-02: Team Role Perception and Leadership Activation under Pressure.

[0077] Therefore, the present invention employs the above-mentioned pilot and flight dispatcher collaborative training method and system, which can break down training barriers, integrate multi-source information, realize deep collaborative decision-making fusion training between pilots and dispatchers, improve the joint decision-making ability of pilots and dispatchers under complex special situations, and significantly enhance aviation safety.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for collaborative training between pilots and flight dispatchers, characterized in that, Includes the following steps: S1: Construct simulated training scenarios that include daily operations or special event scenarios; S2: Based on the simulated training scenario, set training tasks for pilots and flight dispatchers respectively; S3: Establish a multimodal perception training environment, which includes a pilot simulation terminal, a flight dispatcher simulation terminal, and a multimodal data acquisition system for collecting data; the multimodal data includes physiological data, operational behavior data, and voice data of the pilot and flight dispatcher; S4: Pilots and dispatchers collaborate on the same simulated training scenario and collect multimodal data during the operation. S5: The collected operational behavior data and voice data are fused and processed to generate a collaborative decision-making process data stream; the fusion processing includes data reception, time synchronization, feature extraction and feature-level fusion; feature-level fusion is achieved using a deep learning model based on an attention mechanism; the deep learning model based on an attention mechanism is a Transformer model, which aligns the pilot's eye-tracking heatmap with the dispatcher's interface operation feature sequence to generate a decision-response correlation graph and identify decision-response delays or information gaps between the pilot and the dispatcher; S6: The collected physiological indicators of pilots and dispatchers are fused separately to generate their respective physiological characterization data streams; S7: Based on the collaborative decision-making process data flow, a preset performance evaluation model is used to comprehensively evaluate the collaborative performance of pilots and dispatchers, and obtain the performance evaluation results; S8: Based on the collaborative decision-making process data stream, the physiological representation data stream, and the performance evaluation results, the competency indicators and observable behaviors of pilots and dispatchers are analyzed and evaluated using a preset competency evaluation model to obtain competency evaluation results. The quantitative indicators in the performance evaluation results are mapped to preset competency dimensions, which include procedural application, communication, situational awareness, problem solving and decision-making, leadership and teamwork, and workload management. Based on the mapping relationship, the collaborative decision-making process data stream and the physiological representation data stream are correlated and analyzed to obtain the analysis results. Based on the analysis results, a pre-set competency assessment model was used for weighted calculation to decode and infer the knowledge, skills, attitudes, motivations, abilities, and situational cues involved in the observed behavior, and obtain the decoding and inference results. Based on the decoding and inference results, the fuzzy comprehensive evaluation method was used, combined with the AHP (Analytic Hierarchy Process) to assign weights to each competency dimension and its subordinate behavioral indicators, and obtain the competency assessment results. S9: Generate personalized feedback and training reports based on performance evaluation results and competency assessment results.

2. The method for collaborative training of pilots and flight dispatchers according to claim 1, characterized in that, The simulated training scenario is generated and arranged by an artificial intelligence model after performing feature analysis on historical operation and training data.

3. The method for collaborative training of pilots and flight dispatchers according to claim 2, characterized in that, The multimodal data acquisition system is used to acquire: Eye movement data, physiological data, voice data, and operational behavior data from the pilot simulator; Operation log data, voice data, and screen interaction data from the dispatcher's simulation terminal; Aircraft state parameters and scenario event data in simulated flight environments.

4. The method for collaborative training between pilots and flight dispatchers according to claim 3, characterized in that: The quantitative evaluation indicators of the performance evaluation model include decision-making timeliness, operational accuracy, communication effectiveness, and workload. Decision-making timeliness is quantified by one or more indicators among fault identification time, initial solution proposal time, and final decision execution time. Communication effectiveness is quantified by one or more indicators among standard language usage rate, communication repetition completeness rate, and information request-response delay.

5. The method for collaborative training of pilots and flight dispatchers according to claim 4, characterized in that, The performance evaluation model adopts the fuzzy comprehensive evaluation method and combines the AHP (Analytic Hierarchy Process) to determine the weight of each dimension and output a comprehensive performance score.

6. The method for collaborative training of pilots and flight dispatchers according to claim 5, characterized in that, Also includes: Based on performance evaluation results and competency assessment results, the difficulty or task settings of subsequent simulation training scenarios are dynamically adjusted to achieve adaptive training.

7. A pilot-flight dispatcher collaborative training system, employing a pilot-flight dispatcher collaborative training method as described in any one of claims 1-6, characterized in that, include: The scenario generation and management module is used to build, store, and distribute simulation training scenarios; The multimodal data acquisition module is used to collect data from the pilot's terminal, the dispatcher's terminal, and the simulated environment; The data fusion and processing module is used to fuse the collected operational behavior data and voice data to obtain a fused collaborative process data stream, and to fuse the collected physiological indicators of pilots and dispatchers to obtain a fused physiological characterization data stream. The performance evaluation module is used to comprehensively evaluate the collaboration performance based on the fused collaboration process data flow and using a performance evaluation model to obtain the performance evaluation result. The competency assessment module is used to analyze and evaluate competency indicators and observable behaviors based on the fused collaborative process data stream and the fused physiological representation data stream, using a competency assessment model to obtain competency assessment results. The report generation module is used to generate personalized feedback and training reports based on performance evaluation results and competency assessment results.

8. A pilot-dispatcher collaborative training system according to claim 7, characterized in that, Also includes: The adaptive training control module is used to dynamically adjust the difficulty or task settings of subsequent simulation training scenarios based on performance evaluation results and competency evaluation results, so as to achieve adaptive training.

Images