Pilot situational awareness intelligent training method and system

Abstract

The invention relates to the technical field of artificial intelligence and flight training, and particularly discloses an intelligent training method and system for consciousness of a pilot. The method comprises the following steps: constructing a multi-modal flight scene and an original task structure chart, and extracting a flight task target and a semantic tag set; guiding a pilot to carry out natural language expression, collecting a voice stream and combining a task stage to align a text; comparing the language expression with task semantics by using a language model, and identifying expression differences; generating a feedback statement based on the difference, and pushing the feedback statement to the pilot; updating a task flow and reconstructing an execution script; finally, training records are counted, the capacity label weight is updated, and a capacity growth trend graph is generated. According to the method, accurate identification of pilot task understanding deviation and language expression defects can be realized, a scene training process dynamically adaptive to a cognitive state is constructed, and scene awareness and language expression ability of a pilot are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and flight training technology, and in particular to a pilot situational awareness intelligent training method and system. Background Art

[0002] As flight missions continue to increase in complexity, maintaining good situational awareness and language skills in high-pressure, multi-task environments has become a key factor in flight safety. Traditional pilot training systems rely heavily on static scenarios and rule-based assessment methods, making it difficult to accurately identify pilots' expression deviations, comprehension gaps, and cognitive load during mission execution. Furthermore, pilots' natural language expressions during training are often not systematically utilized, lacking a mechanism for analyzing the deep semantic associations between language content and task structure. This results in delayed training feedback and unclear paths for cognitive improvement.

[0003] In recent years, generative language models have demonstrated impressive capabilities in natural language processing, intelligent question answering, and semantic understanding. However, in flight training scenarios, there is still a lack of effective methods for deeply integrating these models with multimodal flight data (such as mission scripts, simulator images, and event logs) to establish a dynamic language cognitive feedback loop. Existing technologies struggle to precisely evaluate and guide pilots' language expressions based on the semantic structure of flight missions, and they also fail to construct training task structures that dynamically adapt to individual cognitive states. Summary of the Invention

[0004] The present invention provides a method and system for intelligent training of pilot situational awareness, aiming to solve the problem of how to accurately analyze task understanding deviations and expression defects based on multimodal flight scenario data, pilot natural language expression, and generative language models, construct a situational training process that dynamically adapts to cognitive states, and improve pilot situational awareness and language expression ability.

[0005] In order to solve the above technical problems, the present invention provides a pilot situational awareness intelligent training method, comprising: S100, generating a multimodal flight scenario and constructing an original mission structure diagram, defining mission phases and scenario events through graph node annotation, and outputting a flight mission objective and a structured mission semantic label set; S200: guiding the pilot to express in natural language based on the flight mission objective and structured task, collecting the pilot's voice stream and transcribing it into text data, and performing preliminary alignment processing of the language content and the task phase by synchronizing timestamps with the scenario log; S300, performing semantic comparison between the text data and the task semantic labels in the structured task semantic label set, analyzing information omissions and deviations in the natural language expression using a language modeling module, and outputting a result of the pilot's language expression differences in the task phase; S400: Generate language feedback content based on the language expression difference result, wherein the language feedback content includes at least feedback sentences including instruction prompts, error correction instructions, and cognitive suggestions, and push the feedback content to the pilot via text and / or voice. S500: updating the original task structure diagram according to the feedback statement, adjusting the node triggering conditions and prompt sequence, and constructing a reconstructed task flow and execution script that matches the current cognitive state; S600: Based on the pilot's natural language expression, task completion, and language expression difference results during the training process, collect training records and update capability label weights to generate subsequent training plans and capability growth trend charts.

[0006] Furthermore, in the step S100, the multimodal flight scenario at least includes image information, mission script information and flight event data generated based on a flight simulator, and constructs a mission structure diagram.

[0007] Furthermore, in the step S200, the pilot's voice stream is transcribed into text data through a voice recognition module and aligned with the mission phase timestamp in the scenario log to form a language expression sequence in the mission context.

[0008] Furthermore, in the step S300, the language modeling module performs contextual semantic matching on the text data and the task semantic tags in the structured task semantic tag set, and outputs language expression difference results that include at least task goal omission points, behavior description missing points and reasoning error points.

[0009] Furthermore, in the step S400, the language feedback content constructs a feedback sentence by calling a language generation module, which includes at least a text prompt and a synthesized speech bimodal output, and the content covers task instruction restatement, thinking logic error correction and cognitive understanding suggestions.

[0010] Furthermore, after S500, the method also includes: obtaining the task node information in the feedback statement and identifying the corresponding original task structure diagram node; adjusting the connection relationship and triggering sequence of the original task structure diagram nodes to generate a reconstructed task structure diagram; constructing a new execution script based on the reconstructed task structure diagram to generate a training process that is adapted to the current cognitive state.

[0011] Furthermore, After S600, the method further includes: integrating the pilot's natural language expression, task completion and language expression difference results into a training data recording unit; extracting indicators and assigning weights to the training data recording unit to complete the initial value update of the capability label; extracting the staged weight change trend from the historical records of the capability label, generating a capability growth trend chart and storing it in a training record database.

[0012] Furthermore, the capability growth trend graph is generated based on the changes in capability label weights in historical training records, and is used to characterize the pilot's cognitive capability evolution path during multiple training sessions.

[0013] Furthermore, the natural language expression, the language expression difference result and the capability label weight are synchronously stored in the training record database for calling by the training management module to perform the next round of training task allocation and evaluation analysis.

[0014] Furthermore, a pilot situational awareness intelligent training system includes: The scenario modeling module is used to generate multimodal flight scenarios and construct the original mission structure diagram, define mission phases and scenario events through graph node annotation, and output flight mission objectives and structured task semantic label sets; A language acquisition module is used to guide the pilot to express himself in natural language based on the flight mission objectives and structured tasks, collect the pilot's voice stream and transcribe it into text data, and complete the preliminary alignment of language content and mission phase by synchronizing timestamps with scenario logs; a semantic comparison module, configured to perform semantic comparison between the text data and the task semantic labels in the structured task semantic label set, analyze information omissions and deviations in the natural language expression using a language modeling module, and output a result of the pilot's language expression differences during the task phase; a feedback generation module, configured to generate language feedback content based on the language expression difference results, wherein the language feedback content includes at least feedback sentences including instruction prompts, error correction instructions, and cognitive suggestions, and push the feedback content to the pilot via text and / or voice; A task reconstruction module is used to update the original task structure diagram according to the feedback statement, adjust the node triggering conditions and prompt sequence, and construct a reconstructed task process and execution script that matches the current cognitive state; The capability assessment module is used to compile statistics on training records and update capability label weights based on the pilot's natural language expression, task completion and language expression difference results during training, so as to generate subsequent training plans and capability growth trend charts.

[0015] The key innovations of the present invention include: (1) Structured flight mission modeling method: A task structure graph modeling method based on graph nodes is proposed to uniformly represent multimodal flight scenarios and mission stages, solving the problem of unstructured and difficult to analyze flight mission data.

[0016] (2) Semantic comparison mechanism between structured task semantic label sets and natural language expressions: Generative language models and semantic embedding matching algorithms are introduced to identify omissions and errors in the expression content and achieve accurate assessment of pilots' cognitive biases.

[0017] (3) Cognitive state-driven task reconstruction mechanism: Generate feedback statements and reconstruct the task structure diagram based on language expression differences, breaking the limitations of traditional fixed training scripts and achieving real-time adaptation to the pilot's ability status.

[0018] (4) Ability label updating and growth trend modeling method: Establish training data recording units and ability growth trend chart generation process to support the staged modeling and quantitative evaluation of cognitive abilities, and improve the interpretability and strategic nature of training effects.

[0019] The following are its main beneficial effects: On the one hand, the present invention generates multimodal flight scenarios and constructs a primitive mission structure diagram in S100, unifying the modeling of image information, mission script information, and event data in the flight simulator. Mission phases and key events are annotated as graph nodes, achieving a structured representation of the mission process and flight context, providing a contextual foundation for subsequent semantic alignment and reasoning analysis. This innovative structured representation model addresses the fragmented mission phase information and difficulty in unified modeling in traditional flight training.

[0020] Furthermore, leveraging the natural language acquisition and expression semantic comparison processes from S200 to S300, the present invention introduces an expression analysis mechanism based on a generative language model. This precisely aligns the time segments of the pilot's speech stream with the flight mission phases, and matches language content with task labels in a semantic embedding space. This allows for the extraction of multi-dimensional differences, such as missing mission objectives, missing behavioral descriptions, and reasoning errors, to accurately identify pilots' weaknesses in cognitive understanding and language expression. The semantic embedding and difference comparison algorithms introduced in this process significantly outperform keyword retrieval and template matching in detecting expression deviations, effectively addressing the shortcomings of existing evaluation methods in terms of depth of understanding and feedback granularity.

[0021] Furthermore, in the feedback generation and task reconstruction process from S400 to S500, the present invention adopts a language generation module to construct a feedback sentence with language feedback content including at least task prompts, cognitive suggestions and error correction instructions, and pushes it to the pilot terminal to achieve personalized training intervention based on expression defects. At the same time, the system dynamically updates the triggering conditions and sequence of the nodes in the original task structure diagram based on the feedback results, generates a reconstructed task process that matches the pilot's current cognitive state, and outputs a new execution script to build a highly adaptable and evolvable training system. This mechanism significantly improves the adaptability and individual targeting of the training plan, breaking through the bottleneck of the traditional fixed task process's slow response to the pilot's growth path.

[0022] Finally, in S600, the system integrates data such as pilots' natural language expressions, task completion, and expression differences into training data recording units. It then extracts indicators and updates competency label weights, continuously generating competency growth trend charts. This enables long-term evaluation and trend visualization of pilot training progress, establishing a closed-loop feedback loop of training, evaluation, and optimization. Compared to traditional training methods based on static scoring, the dynamic competency modeling method of this invention offers stronger continuous tracking and periodic feedback capabilities, helping to support more accurate training task allocation and evaluation analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of the intelligent training method for pilot situational awareness provided in an embodiment of the present application; Figure 2 This is a structural block diagram of the pilot situational awareness intelligent training system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0025] The following describes in detail the pilot situation awareness intelligent training method provided by the embodiment of the present invention, using the pilot situation awareness intelligent training system (hereinafter referred to as the system) as the execution body: Example 1: Reference Figure 1 , is a flow chart of a pilot situational awareness intelligent training method provided by an embodiment of the present invention. The flow chart may include at least S100-S600: S100: Generate a multimodal flight scenario and construct an original mission structure diagram, define mission phases and scenario events through graph node annotation, and output flight mission objectives and structured mission semantic label sets.

[0026] S200: Guide the pilot to express himself in natural language based on the flight mission objectives and structured tasks, collect the pilot's voice stream and transcribe it into text data, and complete the preliminary alignment of language content and mission phase by synchronizing timestamps with scenario logs.

[0027] S300: Perform semantic comparison between the text data and the task semantic labels in the structured task semantic label set, use the language modeling module to analyze information omissions and deviations in natural language expressions, and output the pilot's language expression differences in the task phase.

[0028] S400: Generate language feedback content based on the language expression difference result. The language feedback content includes at least feedback sentences including instruction prompts, error correction instructions, and cognitive suggestions, and is pushed to the pilot via text and / or voice.

[0029] S500: Update the original task structure diagram according to the feedback statement, adjust the node triggering conditions and prompt sequence, and construct a reconstructed task process and execution script that matches the current cognitive state.

[0030] S600: Based on the pilots' natural language expressions, task completion, and language expression differences during training, collect training records and update capability label weights to generate subsequent training plans and capability growth trend charts.

[0031] In some embodiments, the multimodal flight scenario in S100 includes at least image information, mission script information, and flight event data generated based on a flight simulator, and constructs a mission structure diagram. Specifically, S100 includes at least S110-S130: S110 , acquiring image information, mission script information, and flight event data in the flight simulator, performing scene modeling, and obtaining a multimodal flight scenario dataset.

[0032] Specifically, the data acquisition interface of the flight simulator platform is first called to obtain three types of data resources that match the training scenario, namely scene image information (i.e. the above-mentioned image information), mission script information and flight event data.

[0033] Optionally, the scene image information includes at least: instruments, radar, HUD display screens and visual image sequences from the perspective of the flight cabin; the mission script information includes at least: the start and end time of mission execution, stage division rules, mission objectives and decomposition paths, etc.; the flight event data includes at least: time series structure data such as flight speed changes, weather changes, emergencies, and communication interference.

[0034] After acquiring these three types of raw data, the system simultaneously analyzes them using a scenario modeling engine to form a unified multimodal flight scenario dataset. This multimodal flight scenario dataset, constructed with time as the primary axis and using a combination of frame-level annotation and stage labels, features mission event synchronization. This multimodal flight scenario dataset serves as the foundational corpus for task modeling and language alignment in subsequent steps of the present invention.

[0035] Furthermore, during the generation process, the image information and flight event data are indexed at the frame level by time segment and bound to the mission phase identifiers in the mission script information, facilitating the subsequent construction of the mission structure diagram and node binding operations. After this stage is completed, the system's output multimodal flight scenario dataset is passed as input to S120 for flight event logic extraction and mission diagram construction.

[0036] S120. Extract the logical sequence of flight events from the multimodal flight scenario dataset, construct an original task structure diagram, and complete the graph node connection between the flight phases and events.

[0037] In S120, the system extracts key event nodes that occurred during the flight mission execution from the multimodal flight scenario dataset obtained in S110 and constructs the contextual relationships between these key event nodes along the timeline to form an initial mission structure diagram. This mission structure diagram is in the form of a directed graph, in which nodes represent flight phases or subtasks, and edges represent the logical or temporal relationships between tasks.

[0038] Specifically, the system first analyzes the time period tags in the mission script information and, combined with the sudden node information in the flight event data, identifies the corresponding relationship between the start and end times of the flight event and the phases. For example, a mission may include at least a "takeoff phase," a "cruise phase," a "weather avoidance phase," and a "landing phase," each of which can be labeled as a graph node.

[0039] Secondly, the system classifies the event trigger conditions corresponding to each node and extracts the logical dependencies between events, such as whether a certain stage depends on the successful completion of the previous stage, or whether specific flight conditions (such as wind speed, air pressure and other indicator ranges) need to be met before entering the next stage.

[0040] Subsequently, graph nodes and connecting edges are constructed based on the above-mentioned timing and logical information to form the original mission structure diagram. This diagram serves as the semantic skeleton of the flight mission, providing a structural positioning basis for the semantic annotation work in S130 and serving as a carrier for subsequent updating and reconstruction in S500.

[0041] Each node in the task structure diagram includes at least basic information such as the corresponding stage number, flight stage name, trigger condition, corresponding event code, etc., for association and use in subsequent modules.

[0042] S130 , semantically annotating each node in the original mission structure diagram to generate a flight mission target and a structured mission semantic label set.

[0043] S130, the most important step in S100, performs multi-layer semantic annotation on each task node based on the original task structure diagram, outputting a structured task semantic label set and flight mission objectives. This structured task semantic label set will serve as the core reference standard for language expression and task alignment analysis in S300.

[0044] Specifically, the system reads the information of each graph node from the original task structure graph constructed by S120 and performs semantic annotation according to the following dimensions: Mission objective: represents the core tasks that need to be completed at this stage, such as "switch to automatic navigation" or "send weather report". State requirements: Indicates the state conditions that the system must meet when starting this task node, such as "stable flight altitude", "wind speed less than a certain threshold", etc. Operational expectations: Define the key actions that the pilot should take during this phase, such as "pushing the thrust levers forward" and "adjusting the heading angle." Information response: defines the semantic content that the pilot needs to express, including at least confirmation information, voice broadcast or system repetition, etc.

[0045] These semantic labels are structured and bound to nodes, forming data pairs that correspond one-to-one between task nodes and labels. Once all nodes are semantically labeled, the system automatically merges the task semantic labels from the multi-node structured task semantic label set within the task phase to form a structured task semantic label set. This is then combined with the top-level objectives in the task script information to output the flight mission objective.

[0046] Furthermore, the flight mission objectives will serve as the core prompt content of the pilot's language guidance information in S200, and the structured task semantic tag set will serve as the semantic reference set for comparing the language expression with the mission objectives in S300.

[0047] This step provides a clear and structured contextual semantic framework for pilot training by constructing a multimodal task structure model combined with semantic annotation. The generated original task structure diagram not only supports the execution of task script information but also serves as a reference for language comparison and feedback generation, forming a closed data loop in subsequent modules and establishing a foundation for semantic consistency between task driving, language modeling, and behavior recognition. Flight mission objectives and structured task semantic label sets are integrated throughout the training process, providing direct semantic indexing support for semantic feedback generation, capability assessment, and task reconstruction.

[0048] In some embodiments, the pilot's speech stream in S200 is transcribed into text data by a speech recognition module and aligned with the mission phase timestamps in the scenario log to form a language expression sequence in the mission context. Specifically, S200 includes at least S210-S230: S210: Acquire the pilot's voice stream during the flight mission, input it into the voice recognition module for transcription, and obtain text data.

[0049] Specifically, during flight training, the system uses the voice acquisition unit to capture the pilot's voice input in real time, generating a voice stream. This voice stream is collected synchronously with the flight mission execution process and has a timestamp recording function to ensure subsequent temporal alignment with the scenario log.

[0050] During the voice collection process, the system segments the collected voice stream. Each voice segment is bound to the start and end time of the collection, and the corresponding flight status label is recorded, such as the mission phase identification, flight control mode, operation interface type, etc.

[0051] The speech stream is then fed into the speech recognition module for automatic speech recognition processing. The result is text data, a transcribed text sequence, also known as a language expression sequence. To ensure transcription accuracy, the system employs a context-aware decoding mechanism, using the speech segment along with the current mission objective and mission phase semantic labels as auxiliary inputs to improve the recognition accuracy of specialized terms and instructional expressions.

[0052] The above text data will serve as the content input for task context alignment in S220, and is also the basic language content for constructing semantic embedding pairs in S300.

[0053] S220: Extract the timestamp information of the task phase from the scenario log, align it with the time segment of the text data, and obtain preliminary corresponding language expression content.

[0054] It's important to note that the text data generated by S210 is essentially a time-stamped natural language sequence, but it hasn't yet been semantically mapped to the mission phases defined in the mission structure. To establish a consistent correspondence between the content and the flight mission nodes, it's necessary to extract mission phase timestamp information from the flight simulation platform and construct a mission phase timeline.

[0055] Specifically, the scenario log recording module records the execution trajectory of the flight mission script in real time. Optionally, the execution trajectory includes at least the start and end time of each mission phase, the time when the trigger condition is met, the time when the key event occurs, etc. The system extracts the log content and constructs a mission phase time index table.

[0056] Subsequently, the system matches the time segments of the text data in S210 with the task stage time index table, and the corresponding methods include at least: Exact match: The acquisition time of a certain voice segment falls completely within the time range of a certain task stage; Auxiliary reasoning and matching: When a speech segment spans multiple stage time boundaries, its main stage affiliation is inferred through semantic context; Segment matching: Split the cross-stage speech transcription segment into multiple subsequences, each corresponding to multiple task stages.

[0057] Through the above method, the system can achieve a preliminary correspondence between text data and task stages, forming preliminary corresponding language expression content. This content will be merged with the task semantic tags in the structured task semantic tag set in S230 to generate an expression sequence for semantic comparison.

[0058] It is worth noting here that the task stage definition in S220 is completely based on the node boundaries constructed by the original task structure diagram in S100. The consistency of the node start and end times with the scene log requires the timestamp synchronization setting to be completed during the system initialization phase.

[0059] S230: Integrate the relationship between the language expression content and the task stage to generate a language expression sequence in the task context.

[0060] On the basis of completing the alignment of language transcription and task stages, S230 further integrates the logical contextual relationship between the language expression content and the structured task semantic labels, and outputs a structured task context expression sequence.

[0061] Specifically, the system binds the language expression text corresponding to each completed stage to the corresponding task stage node, and establishes language expression annotation information in the task structure diagram. For example, if a piece of language content is determined to belong to the "takeoff stage", the text will be stored in the graph structure as the "language expression fragment" attribute of the "takeoff stage" node.

[0062] Subsequently, the system performs semantic enhancement annotation on the language expression text based on the structured task semantic labels outputted by S130 in S100. The specific methods of the enhanced annotation include at least: Annotation entity alignment: Mapping the key terms in the above natural language expressions with the operation instructions or flight status entities in the task semantic labels; Labeling instruction intent: parsing semantic intent fragments in the language, corresponding to expression intents such as "task confirmation", "status broadcast", and "operation decision"; Marking discourse boundaries: Determine the start and end context of the above natural language expressions to provide contextual boundary conditions for subsequent language generation models.

[0063] After completing the above annotation, the system arranges the language expressions of all bound task stages in chronological order to generate a language expression sequence in the task context. This sequence has the following characteristics: Each expression fragment is bound to a specific task stage node; Each expression segment has completed semantic enhancement of entities, intentions, and boundaries; The order of expression sequences is consistent with the flight mission script execution trajectory; The expression sequence maintains a semantic synchronization relationship with the structured task semantic label set.

[0064] This sequence of natural language expressions will serve as the core language input for constructing "semantic embedding pairs" in the S300 module, and will be used for semantic comparison analysis with structured task semantic labels to identify cognitive biases, omissions or errors in the pilot's natural language expressions.

[0065] In implementation S230, the system maps the pilot's spoken content to the task phase nodes in the original task structure diagram based on the language content and the time index information in the scenario log. Furthermore, the system semantically enhances the pilot's language content using the semantic entities and intent information in the structured task semantic tag set, generating a language sequence within the task context. This sequence provides semantic input support for subsequent difference analysis and language feedback generation.

[0066] This step captures the pilot's natural language expressions during the mission, transcribes them, and establishes a correspondence between them and the mission phases. This generates a structured, time-consistent sequence of natural language expressions, providing a complete and traceable source of semantic data for the semantic comparison and analysis of expressions and missions in subsequent modules. By combining mission objectives with structured mission semantic labels for alignment modeling, a closed-loop temporal relationship and semantic mapping between language content and missions is achieved, providing fundamental data support for the system's scenario-driven intelligent cognitive training.

[0067] In some embodiments, the language modeling module in S300 performs contextual semantic matching on the text data and the task semantic labels in the structured task semantic label set, and outputs language expression difference results including at least missing points of task objectives, missing points of behavior descriptions, and points of reasoning errors. Specifically, S300 includes at least S310-S330: S310: Obtain text data and task semantic labels from a structured task semantic label set, and construct a semantic embedding pair of task semantics and language expression.

[0068] Specifically, the system first obtains the language expression sequence in the task context output in S230. Each expression unit in this natural language expression sequence is bound to a specific task stage node and annotated with time sequence and context. At the same time, the system reads the structured task semantic label set generated by the task structure diagram in S130. This structured task semantic label set has been divided into multiple task sub-goals, semantic entities, and expected expressions according to the task nodes.

[0069] The system then matches each linguistic expression fragment with the semantic label associated with the corresponding task phase, forming a "linguistic expression-task semantics" pair and constructing the semantic embedding input set to be processed. To ensure semantic expression capabilities in subsequent modeling, the system introduces a language modeling module for deep semantic embedding modeling.

[0070] During the semantic embedding phase, the language modeling module encodes expressions using contextual information to ensure consistency at the semantic level. This semantic modeling approach uses vectorized semantic embedding, annotated with the directive and phased nature of task semantics, ensuring that the resulting semantic embedding is capable of recognizing behavioral intent and task context.

[0071] Finally, the set of “semantic embedding pairs” output by S310 will serve as the input of S320 for subsequent semantic matching and difference identification.

[0072] S320: Perform semantic matching and context analysis on the semantic embedding pairs to obtain difference information between the natural language expression and the task semantics.

[0073] Based on S310, this step uses the semantic matching mechanism in the language modeling module to perform multi-dimensional comparison processing on the "language expression-task semantics" embedding pair, mainly identifying missing, offset, errors and other components in the natural language expression that are inconsistent with the task requirements, and obtain semantic difference information.

[0074] Specifically, the system first compares each pair of semantic embedding vectors at the content level, including at least semantic comparison metrics such as vocabulary coverage, entity item mention accuracy, and whether the logical order of instructions matches. If the pilot's natural language expression fails to mention the key terms, status descriptions, or instruction actions required by the aforementioned task semantic labels, it is marked as "missing expression." If the expression content is ambiguous or the order is offset from the task semantics, it is marked as "logical deviation."

[0075] Furthermore, the system incorporates a contextual consistency analysis mechanism, comparing expressions before and after each mission phase to identify inconsistencies such as reasoning jumps, causal inversions, or goal switching. This contextual analysis step is particularly critical, as some expressions in flight missions are derived based on the states of previous and subsequent phases. Their semantic structure is nonlinear, requiring the inference capabilities of language models to discern these inconsistencies.

[0076] In addition, in order to accommodate the interference of non-standard factors such as expression redundancy and differences in expression style, the system also sets a lower limit threshold for similarity confidence. Expression comparison items below the threshold are marked as "requires manual review" for review by the evaluation management module.

[0077] Finally, S320 summarizes all the difference markers identified in the above analysis process and outputs a difference information set as a basis for extracting language expression defects in the next step.

[0078] S330: extract missing points of task objectives, missing points of behavior descriptions, and reasoning errors from the difference information, and output the language expression difference results.

[0079] After the semantic difference information is output, the system needs to refine and categorize the difference types so that more targeted prompts can be constructed for the subsequent language feedback module (i.e., in S400). S330 completes the classification and extraction process and outputs the standardized language expression difference results.

[0080] Specifically, the system processes the difference information outputted in step S320 according to the following three types of difference labels: Missing mission objectives: refers to the pilot's failure to clearly express the flight objectives, mission names, or operation instructions specified in the mission semantic tags in the structured mission semantic tag set in the natural language expression, such as missing "confirm target position" or "broadcast heading information"; Missing behavior description: This refers to the pilot not clearly expressing the action process or environmental status description that should be performed at the current stage, such as "whether the air pressure setting is completed" or "whether the route deviation avoidance is executed"; Reasoning errors: These refer to inversion of cause and effect, misjudgment, or inconsistency in inference logic in the pilot's language. For example, "turning because the radar warning was cleared" conflicts with the semantic requirement of "maintaining the current heading after clearing the warning" in the task semantic tag set of the structured task semantic tag.

[0081] Based on the above classification criteria, the system assigns a unique number to each type of difference content and outputs it in a structured format to the "Language Expression Difference Results" collection. The format of this result collection includes at least: task stage number, difference type, difference segment, suggested matching semantic item, and confidence score.

[0082] The final output of language expression difference results serves as the input basis for S400 to construct language feedback content, ensuring that the feedback generation process has a clear source of expression difference, location of semantic deviation and expression goal orientation.

[0083] This step constructs semantic embedding pairs from "text data to task semantic labels from a structured task semantic label set," and uses the language modeling module to perform contextual semantic comparison, successfully identifying deviations in pilots' language expressions during different mission phases. This differential classification and structured output provides a fine-grained cognitive diagnostic foundation for the system. This ensures that subsequent feedback generation, task reconstruction, and capability assessment processes are supported by accurate and traceable semantic sources, significantly enhancing the training system's intelligent language feedback capabilities and cognitive adaptability.

[0084] In some embodiments, the language feedback content in S400 is constructed by calling the language generation module to construct feedback sentences including at least text prompts and synthesized speech bimodal output, and the content covers task instruction restatement, thinking logic correction and cognitive understanding suggestions. Specifically, S400 includes at least S410-S430: S410: Obtain the language expression difference result as input, and call the language generation module to generate a draft of the feedback content.

[0085] Specifically, the system first obtains the language expression difference results output by S330. These language expression difference results include at least expression difference annotations corresponding to each flight mission phase, specifically including at least three difference identification items: missing mission objectives, missing behavior descriptions, and reasoning errors. Each difference identification item includes at least the following fields: difference type, corresponding mission node number, difference segment, recommended semantic content, and difference confidence score.

[0086] The above-mentioned language expression difference results will serve as the input semantic source for the language generation module. The language generation module uses the difference content as a prompt condition and combines it with the target description of the corresponding task stage in the structured task semantic label set to construct a preliminary draft of the language feedback statement. This preliminary draft of the feedback statement must cover three feedback categories: Command prompts: generated based on missing points in the mission objective, such as "Please add the current heading confirmation instruction"; Error correction instructions: generated based on reasoning errors, such as "Please reconfirm the causal relationship. The current course change should be made after the weather improves." Cognitive suggestion category: generated based on missing points in the behavior description, such as "it is recommended to supplement the specific explanation of the posture adjustment process."

[0087] Understandably, the language generation module not only considers single differences when constructing the first draft, but also utilizes the contextual semantics of the task phase to ensure the semantic coherence of the feedback content. Each first draft of feedback content has a one-to-one mapping relationship with its corresponding difference source, and is accompanied by a feedback category identifier for subsequent formatting.

[0088] The output of S410 is a set of structured feedback content drafts, which are input to S420 for formatting and multimodal conversion.

[0089] S420. Format and multimodally convert the first draft of the feedback content to construct text prompts and synthesized speech output content. The text prompts and synthesized speech output content cover task instruction restatement, logical thinking correction, and cognitive understanding suggestions.

[0090] After obtaining the draft structured feedback content generated in S410, the system first standardizes the content. This process includes at least language cleaning (e.g., removing redundant words and standardizing terminology), paragraph reordering (for clearer logical flow), and task tag matching (ensuring that the feedback content matches the structured task semantic tag set numbering).

[0091] The system then enters the multimodal conversion phase. Based on the system settings, the feedback statement will be converted into both text prompts and synthesized speech output. Specifically, it includes at least: Text prompt generation: The system categorizes and groups standardized feedback statements by feedback type (command prompts, error correction instructions, cognitive suggestions), and generates prompt box content in a hierarchical text structure for pilots to read and confirm; Synthesized speech generation: Based on the text prompt content, the speech synthesis module is called to generate speech content. The speech content must be consistent with the text data and the intonation must be natural and unambiguous. Multimodal binding structure generation: The system binds text prompts and voice prompts to task node numbers and constructs a multimodal feedback object set for subsequent push phases.

[0092] During this process, the system also incorporates a mission phase recognition mechanism to ensure that feedback content is correctly presented on the pilot's audio-visual terminal when the mission node is triggered. Furthermore, all synthesized speech content is accompanied by audio duration information, allowing the push module to adjust the feedback rhythm.

[0093] The output of S420 is multimodal formatted feedback content, including at least a text prompt set and a synthesized voice file set, both of which are bound to a specific task node number and a difference item identifier for use in S430.

[0094] S430: Bind the text prompt and the synthesized voice content to a push channel, and output a feedback statement to the pilot interaction terminal.

[0095] Specifically, after completing the formatting and conversion operations in S420, the system binds each set of text prompts and synthesized voice to a feedback push channel. The push channel includes at least a graphical user interface module and a voice broadcast module, which are deployed on the pilot training interactive terminal.

[0096] The system first identifies the current flight mission execution phase and searches for feedback content related to that node based on the mission phase number. If so, the feedback is pushed as follows: Text prompt push: A prompt box with a difference identification number pops up in the interactive interface. The prompt box content is the standardized feedback text, and the pilot can click to view the details; Synthesized voice push: The voice broadcast module broadcasts the content of each feedback statement in sequence. The voice playback rhythm can be adjusted, paused or replayed through the interface control; Multimodal synchronous control: The system sets up a feedback channel binding control mechanism to ensure that text and voice are triggered at the same time to avoid disordered prompts or missing information.

[0097] To improve interaction efficiency, the system also has a feedback response confirmation mechanism, whereby pilots can select options such as "Aware", "Need further explanation", or "Review". The system will record the feedback interaction results in the training log for S600 evaluation and analysis.

[0098] S430 ultimately achieves the dynamic distribution of feedback statements, allowing pilots to receive timely cognitive prompts related to their own expression deviations during training execution.

[0099] This step implements intelligent feedback for pilots' expression deviations by introducing a language generation module and a multimodal push mechanism. The system constructs precise, personalized, and contextually relevant feedback based on language differences and delivers it in real time via both text and voice channels. This allows pilots to receive instant cognitive correction prompts during mission execution, effectively improving their language expression skills and depth of mission comprehension, providing critical support for subsequent training structure reconstruction and dynamic updating of capability labels.

[0100] In some embodiments, after S500, the method further includes at least S510-S530: S510: Obtain task node information in the feedback statement and identify the corresponding original task structure diagram node.

[0101] Specifically, in this step, the system first extracts task-related information from the verbal feedback sentence output in S400. This verbal feedback sentence is generated jointly by S420 and S430. Optionally, the verbal feedback sentence includes at least task instruction restatement, logical thinking correction, and cognitive understanding suggestions. The text also includes semantic hints for the task node, task number, or related descriptive phrases.

[0102] The original mission structure diagram is the output of S100. The multimodal flight scenario dataset acquired in S110 is analyzed in S120 to form a graph structure, which is then semantically labeled in S130. This structure diagram includes multiple uniquely identified graph nodes, each representing a mission phase or event action, with associated attributes such as trigger conditions, event types, and semantic labels.

[0103] During the recognition process, the system uses keyword matching and semantic parsing to associate the task node information in the feedback sentence with the graph nodes in the original task structure graph. The matching process compares the label definitions in the task semantic label set within the structured task semantic label set to ensure semantic consistency and clear node reference.

[0104] If there are multiple task-related prompts in the feedback statement, the system can extract each node information in chronological order or semantic dependency order, and form a task node mapping list as the input data structure in S520.

[0105] S520: Adjust the node connection relationship and triggering sequence of the original task structure diagram to generate a reconstructed task structure diagram.

[0106] After obtaining the task node corresponding to the feedback statement, S520 further adjusts the connection mode and execution path in the original task structure diagram based on the node.

[0107] Specifically, the system first reads the node connections and trigger conditions defined in the original task structure diagram, including at least the directional attributes of the edges, trigger logic parameters, and sequence control fields. For each identified graph node, the system then makes structural changes to its associated upstream and downstream nodes based on the logical suggestions or cognitive restructuring prompts in the feedback statement.

[0108] For example, if the feedback statement prompts "The description of the task stage is incomplete, and it is recommended to observe the task prompt in advance", the system will advance the triggering order of the corresponding node and modify the edge weight setting or logical judgment condition of its predecessor node; for example, if the feedback statement points out that there is an error in the reasoning logic of a certain node, the system can add an explanatory intermediate node after the node to insert a supplementary explanation process to form a temporary cognitive filling structure.

[0109] The update relationship between all nodes retains the main structural clues of the original task stage so that an executable process can be generated in S530. At the same time, the cognitive adjustment type field is marked in the new diagram, such as "rearrangement", "supplementation", "explanation" and other types, for subsequent module recognition processing.

[0110] In the final generated task structure diagram, each node not only carries the original flight mission information, but also includes at least the execution logic change instructions guided by the pilot's cognitive state. This diagram structure is the reconstructed task structure diagram.

[0111] S530: Construct a new execution script based on the reconstructed task structure diagram to generate a training process that is adapted to the current cognitive state.

[0112] After generating the reconstructed task structure graph, S530 constructs an execution script for the training process based on the task nodes and connections in the graph structure. The execution script is a standardized task file that guides the flight simulation training system to push task instructions, prompt statements, and feedback content in sequence according to the graph structure.

[0113] Each task item in the execution script consists of the attributes of a graph node, including at least the task name, trigger condition, prompt type, execution wait time, number of failed retries, and instruction push method. The task name and semantic tags correspond one-to-one with the structured task semantic tag set to ensure structural consistency of the script content.

[0114] During the script construction process, the system also analyzes the language patterns in the feedback sentences and adds multimodal feedback structures, such as graphic displays, audio instructions or dynamic animations, to strengthen the pilot's cognitive correction of deviation knowledge points.

[0115] Understandably, the execution script not only retains the core path of the task process constructed in S100, but also integrates the pilot's individualized cognitive feedback generated in S400, thereby achieving dynamic adaptation and cognitive support of the training process.

[0116] The script is ultimately loaded and called through the task management module as the basic file for a new round of task push on the training platform. It is also linked to the task completion indicator in S600 and used for training evaluation records and capability label weight updates.

[0117] Through the operation of the above-mentioned S500, the original flight mission structure diagram can be updated at the structural level based on the pilot's language expression difference results obtained in the previous module, and the training process can be reconstructed in combination with cognitive restructuring suggestions to generate an execution script that matches the pilot's current cognitive state. While maintaining the core objectives of the mission unchanged, the training process structure can be dynamically adjusted to enhance the system's adaptability and interactive guidance capabilities, forming a flight training closed loop for adaptive optimization of cognitive states.

[0118] In some embodiments, after S600, the method further includes at least S610-S630: S610: Based on the pilot's natural language expression, task completion and language expression difference results, they are integrated into a training data recording unit.

[0119] Specifically, the system first uses the language expression sequence in the task context obtained from S230 as language expression information input, and further combines it with the language expression difference results output by S330. The language expression results have clearly marked multi-dimensional semantic deviation items such as missing points of task goals, missing points of behavior descriptions, and reasoning errors.

[0120] At the same time, combined with the reconstructed mission execution script generated by S500, the pilot's performance in completing each mission phase is evaluated to obtain the mission completion degree. This degree of mission completion is quantified based on the achievement status of each node in the mission script, the accuracy of flight control operations, and the timing of operation response.

[0121] It should be noted that the three core data types—natural language expression sequences, language expression difference results, and task completion—serve as input data sources. These data are integrated according to task chronological order and node hierarchical structure to construct a unified data record format. The data from each round of training is stored as structured training data record units according to fields such as flight phase identifier, semantic tag sequence number, language deviation level, and completion score. These units are then written into the training record database to provide data support for subsequent capability assessments.

[0122] This step is consistent with the data output of stages S200 to S500, especially closely connected with the task stage timestamp in S220 and the semantic difference information analysis logic in S320, ensuring the temporal consistency and semantic integrity of the language behavior data and the task execution process.

[0123] S620: Extract indicators and assign weights to the training data recording units to complete the initial value update of the capability label.

[0124] After the training data record unit is constructed, the system enters the capability label evaluation phase. Specifically, feature extraction is first performed on each training record unit. This process extracts metrics such as keyword density, language logic integrity, and terminology consistency from natural language expressions; extracts metrics such as stage achievement rate, temporal response accuracy, and process execution stability from task completion; and extracts the number of semantic deviations, error type distribution, and cognitive reasoning failure nodes from language expression difference results.

[0125] The aforementioned indicators are categorized and mapped to a pre-defined label structure within the capability labeling system. This system includes at least language comprehension, task-cognition matching, instruction response, semantic reasoning, stage-specific task completion, and information integration. The system maps these indicators based on their weighted proportions and calculates the corresponding scores, ultimately completing the initial weighting update for each capability label.

[0126] The weight allocation strategy is based on a label mapping table and weight distribution matrix defined by experts, dynamically adjusting the sensitivity of each capability dimension. The number of deviation items and performance indicator values in the training data record unit directly influence the weighted results of the capability label, ensuring interpretability and traceability.

[0127] Furthermore, during the weight update process, the system supports multi-label fusion of task category dimensions, that is, independent capability assessment channels are set for specific scenario tasks (such as emergency response tasks, navigation management tasks, etc.), forming a capability weight matrix based on the dual dimensions of task type and capability item.

[0128] S630: Extract the phased weight change trend from the historical records of the capability tags, generate a capability growth trend graph, and store it in the training record database.

[0129] After updating the initial weights of the capability tags, the system automatically retrieves records of each round of capability tags during the pilot's historical training cycle and analyzes their phased changes. Specifically, the system aggregates the historical weights of each tag by training round and mission phase, and constructs a time series tag growth curve.

[0130] During trend extraction, the system can group and analyze statistics based on task categories, extracting the evolutionary characteristics of capabilities within the same task scenario. For critical task modules that undergo repeated training, the system further analyzes the fluctuation range and frequency of changes in their training label weights to assess the stability and growth rate of the training process.

[0131] The system then presents the capability growth trend chart as a graph, with each trend curve corresponding to the score evolution path of a capability tag over multiple training cycles. The trend chart supports bidirectional binding with training data record units and is displayed in real time through an interactive interface. Ultimately, the capability growth trend chart is stored in the training record database, becoming a core reference for subsequent personalized training allocation and phase evaluation in the training management module.

[0132] In some embodiments, the aforementioned capability growth trend chart is generated based on the changes in capability label weights in historical training records, representing the evolutionary path of a pilot's cognitive abilities over multiple training sessions. Specifically, the capability growth trend chart is constructed based on the structured semantic label set (S130), the sequence of natural language expressions (S230), and the language expression difference results (S330) outputted in the previous steps. This chart not only reflects individual pilot capability changes but also maps overall mission adaptability and contextual semantic mastery.

[0133] In some embodiments, the above-mentioned natural language expressions, language expression difference results and ability label weights are synchronously stored in the training record database for the training management module to call for the next round of training task allocation and evaluation analysis.

[0134] This step is the final data collection and processing step of the method of the present invention. Specifically, the natural language expression sequence, language expression difference results, and updated ability label weights generated in the above steps are standardized and coded, and uniformly identified according to training rounds, task stages, and ability label numbers.

[0135] This data synchronization process utilizes a dual binding mechanism of mission number and user ID, ensuring traceability of each pilot's behavior and performance status at each training stage. When the three core training data are written to the training record database, they are logically linked to the generated reconstructed mission structure diagram and execution script, ensuring the continuity and logical consistency of training mission configuration.

[0136] The aforementioned data synchronization results can be directly accessed by the training management module, enabling intelligent allocation of training tasks, intelligent identification of capability gaps, and periodic assessment output. The system supports targeted task allocation based on trends in label weights, and can also push targeted feedback based on variance results, creating a more targeted training closed loop.

[0137] During this process, the data organization structure executed in this step is consistent with the training data recording unit generated by S610, and combined with the ability growth trend chart generated by S630, it realizes the three-dimensional fusion of training data, ability changes and individual behavior, providing efficient support for subsequent training effect feedback and program optimization.

[0138] Furthermore, it should be noted that through the design and implementation of the aforementioned S600, the present invention enables multi-dimensional training record integration and capability quantification analysis based on pilots' natural language expression behaviors, task completion status, and language semantic deviation data. While ensuring logical and semantic consistency between previous and subsequent tasks, the weights of pilot capability labels are dynamically updated, further generating a visual capability growth map based on evolutionary trends, providing a scientific basis for subsequent personalized training recommendations and cognitive level assessments. The module's structure closely aligns with stages S100 to S500, maintaining data logic consistency with language acquisition, semantic comparison, and feedback generation, thus establishing a complete intelligent training closed-loop system.

[0139] In summary, the key innovations of the present invention include: (1) Structured flight mission modeling method: A task structure graph modeling method based on graph nodes is proposed to uniformly represent multimodal flight scenarios and mission stages, solving the problem of unstructured and difficult to analyze flight mission data.

[0140] (2) Semantic comparison mechanism between structured task semantic label sets and natural language expressions: Generative language models and semantic embedding matching algorithms are introduced to identify omissions and errors in the expression content and achieve accurate assessment of pilots' cognitive biases.

[0141] (3) Cognitive state-driven task reconstruction mechanism: Generate feedback statements and reconstruct the task structure diagram based on language expression differences, breaking the limitations of traditional fixed training scripts and achieving real-time adaptation to the pilot's ability status.

[0142] (4) Ability label updating and growth trend modeling method: Establish training data recording units and ability growth trend chart generation process to support the staged modeling and quantitative evaluation of cognitive abilities, and improve the interpretability and strategic nature of training effects.

[0143] The following are its main beneficial effects: On the one hand, the present invention generates multimodal flight scenarios and constructs a primitive mission structure diagram in S100, unifying the modeling of image information, mission script information, and event data in the flight simulator. Mission phases and key events are annotated as graph nodes, achieving a structured representation of the mission process and flight context, providing a contextual foundation for subsequent semantic alignment and reasoning analysis. This innovative structured representation model addresses the fragmented mission phase information and difficulty in unified modeling in traditional flight training.

[0144] Furthermore, leveraging the natural language acquisition and expression semantic comparison processes from S200 to S300, the present invention introduces an expression analysis mechanism based on a generative language model. This precisely aligns the time segments of the pilot's speech stream with the flight mission phases, and matches language content with task labels in a semantic embedding space. This allows for the extraction of multi-dimensional differences, such as missing mission objectives, missing behavioral descriptions, and reasoning errors, to accurately identify pilots' weaknesses in cognitive understanding and language expression. The semantic embedding and difference comparison algorithms introduced in this process significantly outperform keyword retrieval and template matching in detecting expression deviations, effectively addressing the shortcomings of existing evaluation methods in terms of depth of understanding and feedback granularity.

[0145] Furthermore, in the feedback generation and task reconstruction process from S400 to S500, the present invention adopts a language generation module to construct a feedback sentence with language feedback content including at least task prompts, cognitive suggestions and error correction instructions, and pushes it to the pilot terminal to achieve personalized training intervention based on expression defects. At the same time, the system dynamically updates the triggering conditions and sequence of the nodes in the original task structure diagram based on the feedback results, generates a reconstructed task process that matches the pilot's current cognitive state, and outputs a new execution script to build a highly adaptable and evolvable training system. This mechanism significantly improves the adaptability and individual targeting of the training plan, breaking through the bottleneck of the traditional fixed task process's slow response to the pilot's growth path.

[0146] Finally, in S600, the system integrates data such as pilots' natural language expressions, task completion, and expression differences into training data recording units. It then extracts indicators and updates competency label weights, continuously generating competency growth trend charts. This enables long-term evaluation and trend visualization of pilot training progress, establishing a closed-loop feedback loop of training, evaluation, and optimization. Compared to traditional training methods based on static scoring, the dynamic competency modeling method of this invention offers stronger continuous tracking and periodic feedback capabilities, helping to support more accurate training task allocation and evaluation analysis.

[0147] The following is a detailed description of the pilot situational awareness intelligent training system: Example 2: Figure 2 FIG. 1 is a schematic diagram showing the structure of an intelligent training system for pilot situational awareness according to an embodiment of the present invention. Figure 2 As shown, the system may include: The scenario modeling module 10 is used to generate multimodal flight scenarios and construct an original mission structure diagram, define mission phases and scenario events through graph node annotation, and output flight mission objectives and structured mission semantic label sets. Specifically, the scenario modeling module 10 includes: Image information acquisition unit: acquires image information related to the current training scenario from the flight simulator in real time to construct a visual multimodal input of the flight environment; Task script parsing unit: parses task script information, identifies task setting phase, subtask name and triggering conditions; Flight event analysis unit: extracts key event data during flight, including flight attitude changes, instrument operation behavior, and emergency response content; Mission structure graph generation unit: This unit constructs an original mission structure graph based on the multimodal data. Nodes in the graph represent flight phases and events, and edges represent phase transitions. Semantic labeling unit: semantically annotates each node in the task structure diagram to obtain the task semantic label set and flight mission target information in the structured task semantic label set for use by downstream modules.

[0148] The language acquisition module 20 is used to guide the pilot to express themselves in natural language based on the flight mission objectives and structured tasks, collect the pilot's voice stream and transcribe it into text data, and complete the preliminary alignment of the language content and the mission phase by synchronizing the timestamps in the scene log. Specifically, the language acquisition module 20 includes: Voice acquisition unit: acquires the pilot's voice input in real time during the mission; Speech transcription unit: transcribes pilot voice input into text through the speech recognition module; Task log alignment unit: calls the task phase timestamp information in the scene log and aligns it with the time segment of the speech and text data; Language context integration unit: Based on the time alignment results, the pilot's speech expression is semantically associated with the mission phase to form a language expression sequence in the mission context.

[0149] The semantic comparison module 30 is configured to perform a semantic comparison between the text data and the task semantic labels in the structured task semantic label set, analyze the information omissions and deviations in the natural language expression using the language modeling module, and output the difference results of the pilot's language expression during the task phase. Specifically, the semantic comparison module 30 includes: Semantic embedding construction unit: embeds the sequence of natural language expressions and the task semantic labels in the structured task semantic label set into the semantic space respectively to construct semantic embedding pairs; Semantic matching analysis unit: Based on contextual semantic modeling, it identifies the difference between the natural language expression and the task semantics; Deviation extraction unit: further extracts missing task objectives, missing behavior descriptions, and reasoning errors from the difference information; Language expression difference output unit: outputs the language expression difference results obtained from the above analysis for use by the feedback module.

[0150] The feedback generation module 40 is configured to generate language feedback content based on the language expression difference results. The language feedback content includes at least feedback sentences including instruction prompts, error correction instructions, and cognitive suggestions, and push them to the pilot via text and / or voice. Specifically, the feedback generation module 40 includes: Language feedback generation unit: inputs the language expression difference results and calls the language generation module to generate the first draft of the feedback content; Content multimodal conversion unit: formats the draft feedback content into visual text and voice output; Feedback push unit: Pushes feedback statements to the pilot interaction terminal in a dual-modal manner of text, graphics and voice, improving the pilot's understanding and acceptance of training feedback.

[0151] The task reconstruction module 50 is used to update the original task structure diagram according to the feedback statement, adjust the node trigger conditions and prompt sequence, and construct a reconstructed task process and execution script that matches the current cognitive state. Specifically, the task reconstruction module 50 includes: Task node identification unit: parses the task node information involved from the feedback statement; Structure diagram adjustment unit: updates the node triggering conditions and edge connection order in the task structure diagram; Script reconstruction unit: Generates a new task execution script based on the adjusted task structure diagram and feeds it back to the training system scheduling center.

[0152] The capability assessment module 60 is used to collect training records and update capability label weights based on the pilot's natural language expression, task completion, and language expression difference results during training, so as to generate subsequent training plans and capability growth trend charts. Specifically, the capability assessment module 60 includes: Training data integration unit: Based on the pilot's natural language expression, task completion and language expression difference results, it is integrated into a training data recording unit; Capability label update unit: extracts multi-dimensional indicators and assigns weights to the training data recording unit to complete the initial value update of the capability label; Trend modeling unit: Analyzes the historical weight change trend of capability labels and constructs capability growth trend charts; Database synchronization unit: Synchronizes and stores the above natural language expressions, difference results and label weights into the training record database for use by the training management module.

[0153] Optionally, the multimodal flight scenario includes at least image information, mission script information and flight event data generated based on a flight simulator, and constructs a mission structure diagram.

[0154] Optionally, the pilot's voice stream is transcribed into text data through a speech recognition module and aligned with the mission phase timestamps in the scenario log to form a language expression sequence in the mission context.

[0155] Optionally, the language modeling module performs contextual semantic matching on the text data and the task semantic labels in the structured task semantic label set, and outputs language expression difference results that include at least task goal omission points, behavior description missing points and reasoning error points.

[0156] Optionally, the language feedback content constructs a feedback sentence by calling a language generation module, which includes at least a text prompt and a synthesized speech bimodal output, and the content covers task instruction restatement, thinking logic error correction and cognitive understanding suggestions.

[0157] Optionally, the task reconstruction module 50 is also used to obtain the task node information in the feedback statement and identify the corresponding original task structure diagram node; adjust the connection relationship and trigger sequence of the original task structure diagram nodes to generate a reconstructed task structure diagram; and construct a new execution script based on the reconstructed task structure diagram to generate a training process that is adapted to the current cognitive state.

[0158] Optionally, the capability assessment module 60 is further configured to integrate the pilot's natural language expression, task completion and language expression difference results into a training data recording unit; extract indicators and assign weights to the training data recording unit to complete the initial value update of the capability label; extract the phased weight change trend from the historical records of the capability label, generate a capability growth trend chart and store it in the training record database.

[0159] Optionally, the capability growth trend graph is generated based on the changes in capability label weights in historical training records, and is used to characterize the pilot's cognitive capability evolution path during multiple training sessions.

[0160] Optionally, the natural language expression, the language expression difference result and the capability label weight are synchronously stored in the training record database for the training management module to call for the next round of training task allocation and evaluation analysis.

[0161] The pilot situational awareness intelligent training system provided by this invention, by building a closed-loop link of "situation-language-structure-feedback-cognition", has significant beneficial effects in the following aspects: (1) Improved scenario-task alignment: The system effectively restores complex mission environments and improves the accuracy of language expression context through multimodal flight scenario modeling and mission structure diagram generation; (2) Intelligent enhancement of language understanding: With the help of language modeling and semantic comparison modules, omissions, errors and deviations in pilots' natural language expressions can be automatically identified; (3) Intelligent feedback generation upgrade: The system generates personalized correction sentences based on language deviation results, making the training feedback content adaptable and targeted; (4) Dynamic adaptation of task processes: By reconstructing the original task structure, the system can dynamically generate matching training processes based on cognitive states; (5) Visual management of capability evolution: The system collects training data and continuously updates capability label weights, generates growth trend charts, and realizes dynamic quantitative management of pilots’ cognitive capabilities; (6) Improve training efficiency and individual adaptability: The system realizes a closed loop of the entire process from expression collection, error recognition, content correction to capability assessment, significantly improving the efficiency of pilot situational awareness training and intelligent adaptability.

Claims

1. A method for intelligent training of pilot situational awareness, characterized in that: The steps include: S100, generating a multimodal flight scenario and constructing an original mission structure diagram, defining mission phases and scenario events through graph node annotation, and outputting a flight mission objective and a structured mission semantic label set; S200: guiding the pilot to express in natural language based on the flight mission objective and structured task, collecting the pilot's voice stream and transcribing it into text data, and performing preliminary alignment processing of the language content and the task phase by synchronizing timestamps with the scenario log; S300, performing semantic comparison between the text data and the task semantic labels in the structured task semantic label set, analyzing information omissions and deviations in the natural language expression using a language modeling module, and outputting a result of the pilot's language expression differences in the task phase; S400: Generate language feedback content based on the language expression difference result, wherein the language feedback content includes at least feedback sentences including instruction prompts, error correction instructions, and cognitive suggestions, and push the feedback content to the pilot via text and / or voice. S500: updating the original task structure diagram according to the feedback statement, adjusting the node triggering conditions and prompt sequence, and constructing a reconstructed task flow and execution script that matches the current cognitive state; S600: Based on the pilot's natural language expression, task completion, and language expression difference results during the training process, collect training records and update capability label weights to generate subsequent training plans and capability growth trend charts.

2. The method according to claim 1, characterized in that In the step S100, the multimodal flight scenario at least includes image information, mission script information and flight event data generated based on a flight simulator, and a mission structure diagram is constructed.

3. The method according to claim 1, characterized in that In the step S200 , the pilot's voice stream is transcribed into text data through a voice recognition module and aligned with the mission phase timestamps in the scenario log to form a language expression sequence in the mission context.

4. The method according to claim 1, wherein In the step S300, the language modeling module performs contextual semantic matching on the text data and the task semantic tags in the structured task semantic tag set, and outputs language expression difference results that at least include task goal omission points, behavior description missing points and reasoning error points.

5. The method according to claim 1, wherein In the step S400, the language feedback content is constructed by calling a language generation module to construct a feedback sentence including at least a text prompt and a synthesized speech bimodal output, and the content covers task instruction restatement, thinking logic error correction and cognitive understanding suggestions.

6. The method according to any one of claims 1 to 5, characterized in that After S500, the method further includes: Obtaining task node information in the feedback statement and identifying the corresponding original task structure diagram node; Adjusting the node connection relationship and triggering sequence of the original task structure diagram to generate a reconstructed task structure diagram; A new execution script is constructed based on the reconstructed task structure diagram to generate a training process that is adapted to the current cognitive state.

7. The method according to any one of claims 1 to 5, characterized in that After S600, the method further includes: Integrate the pilot's natural language expression, task completion and language expression difference results into a training data recording unit; Extracting indicators and assigning weights to the training data recording units to complete the initial value update of the capability labels; The phased weight change trend is extracted from the historical records of the capability tags, and a capability growth trend graph is generated and stored in the training record database.

8. The method according to any one of claims 1 to 5, characterized in that The capability growth trend graph is generated based on the changes in capability label weights in historical training records, and is used to represent the pilot's cognitive capability evolution path during multiple training sessions.

9. The method according to claim 7, characterized in that The natural language expression, the language expression difference result and the capability label weight are synchronously stored in the training record database for calling by the training management module to perform the next round of training task allocation and evaluation analysis.

10. An intelligent pilot situational awareness training system, characterized in that: include: The scenario modeling module is used to generate multimodal flight scenarios and construct the original mission structure diagram, define mission phases and scenario events through graph node annotation, and output flight mission objectives and structured task semantic label sets; A language acquisition module is used to guide the pilot to express himself in natural language based on the flight mission objectives and structured tasks, collect the pilot's voice stream and transcribe it into text data, and complete the preliminary alignment of language content and mission phase by synchronizing timestamps with scenario logs; a semantic comparison module, configured to perform semantic comparison between the text data and the task semantic labels in the structured task semantic label set, analyze information omissions and deviations in the natural language expression using a language modeling module, and output a result of the pilot's language expression differences during the task phase; a feedback generation module, configured to generate language feedback content based on the language expression difference results, wherein the language feedback content includes at least feedback sentences including instruction prompts, error correction instructions, and cognitive suggestions, and push the feedback content to the pilot via text and / or voice; A task reconstruction module is used to update the original task structure diagram according to the feedback statement, adjust the node triggering conditions and prompt sequence, and construct a reconstructed task process and execution script that matches the current cognitive state; The capability assessment module is used to compile statistics on training records and update capability label weights based on the pilot's natural language expression, task completion and language expression difference results during training, so as to generate subsequent training plans and capability growth trend charts.

Images