Aviation man-machine cooperation task distribution system and method based on cognitive load prediction

Abstract

The invention provides an aviation man-machine cooperation task distribution system and method based on cognitive load prediction. And by dynamically optimizing task allocation, the cognitive load of the pilot is reduced, and the task execution efficiency and the flight safety are improved. According to the system, through multi-dimensional data fusion, intelligent algorithm optimization and man-machine interaction innovation, accurate monitoring and dynamic response to the cognitive state of the pilot are achieved, and the intelligent level of aviation task execution is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of aviation human-machine collaboration technology, specifically relating to an aviation human-machine collaboration task allocation system and method based on cognitive load prediction. Background Technology

[0002] With the rapid development of aviation technology, the complexity and diversity of modern aviation missions have increased significantly. Pilots need to process massive amounts of heterogeneous data from multiple sources, including airborne systems, air traffic control communications, meteorological information, navigation data, and control decisions, while also making rapid and accurate decisions in complex and ever-changing external environments and emergency situations. This poses unprecedented challenges to pilots' cognitive abilities. Traditional aviation human-machine collaborative systems mostly adopt a fixed task allocation model, that is, assigning tasks according to preset task priorities and operating procedures. However, this rigid human-machine interaction model has obvious drawbacks: First, it cannot adapt to the dynamically changing cognitive state of pilots during mission execution, easily leading to cognitive overload or waste of resources; second, fixed task allocation strategies are difficult to cope with sudden situations and rapid changes in the mission environment; finally, this model ignores the impact of individual pilot differences on mission execution effectiveness. These problems seriously restrict the safety and efficiency of aviation missions.

[0003] In actual flight missions, pilots' cognitive load is influenced by a complex interplay of multiple factors. From a mission perspective, mission complexity, information density, time pressure, and the need for multitasking directly determine the level of cognitive load. From an environmental perspective, external factors such as weather conditions (e.g., turbulence, visibility), airspace situation (e.g., air traffic density, distribution of threat targets), and electromagnetic environment (e.g., communication interference, radar signal strength) significantly affect cognitive load. From an individual perspective, pilots' physiological state (e.g., fatigue level, stress level, circadian rhythm), psychological characteristics (e.g., working memory capacity, attention allocation ability, emotional stability), and professional skills (e.g., mission experience, emergency response capabilities) all have a significant impact on cognitive load. When cognitive load exceeds a pilot's capabilities, it can lead to operational errors, delayed reactions, or even mission failure. Research indicates that in complex aviation missions, over 60% of human errors are related to inadequate cognitive load management. Therefore, how to monitor pilots' cognitive state in real time and optimize task allocation based on its dynamic changes has become a critical scientific problem urgently needing to be solved in the field of human-machine collaboration in aviation.

[0004] Existing cognitive load monitoring technologies have the following limitations: First, traditional task allocation systems lack intelligence and adaptability, and cannot dynamically adjust according to the pilot's real-time cognitive state. Although some studies have attempted to introduce machine learning algorithms for task allocation optimization, these methods often have the following problems: (1) the model training data differs from the actual task environment, resulting in insufficient generalization ability; (2) they ignore the impact of individual pilot differences on task performance; (3) they lack adaptability to dynamic changes in the task environment; (4) they fail to effectively integrate multi-source heterogeneous data (such as physiological data, task data, and environmental data) for comprehensive analysis; and (5) they lack real-time performance and predictability, making it difficult to provide early warning and intervention before cognitive load overload. These problems severely restrict the application effect of existing methods in actual aviation missions. Summary of the Invention

[0005] The purpose of this invention is to address the problem that fixed task allocation modes in existing aviation human-machine collaborative systems cannot adapt to the dynamic cognitive state of pilots. It proposes an aviation human-machine collaborative task allocation system and method based on cognitive load prediction. By dynamically optimizing task allocation, this system reduces the cognitive load on pilots, improving task execution efficiency and flight safety. Through multi-dimensional data fusion, intelligent algorithm optimization, and innovative human-machine interaction, the system achieves accurate monitoring and dynamic response to the pilot's cognitive state, significantly enhancing the intelligence level of aviation mission execution.

[0006] According to a first aspect of the present invention, an aviation human-machine collaborative task allocation system based on cognitive load prediction is proposed, comprising a load prediction module, a task arbitration module, and an adaptive training module. The load prediction module establishes a pilot cognitive model and integrates multi-source heterogeneous data, including pilot physiological data (such as heart rate variability, EEG spectral characteristics, eye-tracking data, and skin conductance), flight history data (such as flight experience, historical mission performance, and emergency handling records), and personalized characteristic data (such as reaction speed, decision-making style, spatial cognitive ability, and stress tolerance), to achieve dynamic monitoring and prediction of the pilot's cognitive state. The task arbitration module prioritizes tasks based on a dynamic task priority matrix, evaluating them from three dimensions: safety, timeliness, and operational complexity, and achieves a smooth transition in task allocation through a multimodal interaction system. The adaptive training module generates personalized training plans based on cognitive load characteristics and optimizes training effects through a stress scenario simulation system, improving the pilot's ability to cope with complex environments.

[0007] In one possible embodiment, the load prediction module employs an LSTM-RNN-based load trend prediction algorithm, combined with an attention mechanism and a graph convolutional network, to predict cognitive load trends in a rolling manner with a 5-second time window. It can predict changes in pilots' cognitive load 5-10 seconds in advance, achieving a prediction accuracy of 92.3%.

[0008] In one possible embodiment, the LSTM-RNN algorithm constructs a cognitive load prediction model by collecting pilot physiological data (such as heart rate, EEG, and eye movement data), flight mission data (such as mission complexity, information density, and time pressure), and personality characteristic data (such as decision-making style, reaction speed, and spatial cognitive ability) in real time, and outputs the load change trend. This model achieves accurate prediction of cognitive load through multi-source data fusion and spatiotemporal feature extraction, and supports dynamic adjustment of prediction thresholds to adapt to different mission environments.

[0009] In one possible embodiment, the task arbitration module includes a dynamic task priority matrix, which evaluates the priority of tasks based on three dimensions: security (task failure risk coefficient, system criticality index, environmental impact factor), timeliness (remaining operation time margin, task urgency factor, time pressure index), and operation complexity (number of required operation steps, interface switching frequency, cognitive resource consumption rate). The priority weight of each task is calculated by fuzzy hierarchical analysis (FAHP) and entropy method to generate an optimal dynamic task sequence and update the task queue in real time.

[0010] In one possible embodiment, the mission arbitration module has developed a hybrid command assignment system that combines voice interaction and tactile feedback. The voice interaction system uses a natural language generation engine based on the Transformer architecture to generate commands that conform to the pilot's cognitive habits. The tactile feedback device is integrated into the control stick and outputs encoded tactile signals through a high-frequency vibration module to ensure that the pilot can seamlessly receive mission switching information within 0.5 seconds, avoiding distraction or operational errors due to a single mission assignment method.

[0011] In one possible embodiment, the task arbitration module includes an autopilot system takeover degree dynamic adjustment algorithm that supports a gradual switch from fully manual to fully automatic. The algorithm is based on deep reinforcement learning and Markov decision process (MDP) and adjusts the autopilot system takeover degree at a rate of 5% per second according to the pilot's real-time cognitive load and task requirements. When the cognitive load exceeds a threshold, the system gradually increases the autopilot takeover ratio. When the load decreases, manual operation permission is restored at a rate of 3% per second, achieving a dynamic balance in human-machine collaboration.

[0012] In one possible embodiment, the adaptive training module generates personalized training plans based on cognitive load feature analysis to optimize the pilot's mission execution capabilities; through transfer learning algorithms, the system can identify the pilot's weaknesses in cognitive load and design targeted training content to improve the pilot's performance in high-load tasks.

[0013] In one possible embodiment, the adaptive training module includes a stress scenario simulation system for simulating various coupled scenarios of sudden failures and high cognitive load, such as engine failure, sudden weather changes, system alarms, and communication interruptions. The system monitors the pilot's stress response patterns in real time using devices such as eye trackers, EEG caps, and electromyography (EMG), and combines this with machine learning algorithms to identify behavioral characteristics under stress, dynamically adjusting the training difficulty and scenario complexity to improve the pilot's ability to cope in complex environments.

[0014] In one possible embodiment, the adaptive training module develops metacognitive monitoring technology, which collects pilots' self-assessment data (such as subjective workload scores and situational awareness questionnaires from the NASA-TLX scale) through an intelligent interactive interface and compares and analyzes the data with the system's assessment results. Using a Bayesian optimization algorithm, the system can dynamically optimize the training program, helping pilots better understand their own state, correct self-cognitive biases, and improve their ability to cope with complex flight missions. In addition, the training data adopts a storage mechanism based on blockchain technology to ensure data security and traceability.

[0015] According to a second aspect of the present invention, a method for assigning human-machine collaborative tasks in aviation based on cognitive load prediction is proposed, employing the aforementioned system for assigning human-machine collaborative tasks in aviation based on cognitive load prediction, comprising the following processes: Predict the pilot's cognitive load using the load prediction module; When it is predicted that the cognitive load will exceed the preset threshold within the next 5-10 seconds, the task arbitration module is immediately activated; The task arbitration module generates the optimal dynamic task sequence and updates the task queue in real time; Based on the priority order of each task in the task queue, instructions are transmitted to the pilot through a multimodal interaction channel to ensure that the pilot receives task switching information seamlessly within 0.5 seconds. At the same time, based on the predicted cognitive load of the pilot, a dynamic adjustment algorithm for the degree of takeover of the autopilot system is adopted to achieve a gradual switch from fully manual to fully automatic.

[0016] In one possible embodiment, the specific process of the gradual switch from fully manual to fully automatic includes: The system adjusts the takeover level of the autonomous driving system at a rate of 5% per second; when the cognitive load exceeds the threshold, the system gradually increases the autonomous driving takeover ratio; when the load decreases, manual operation permissions are restored at a rate of 3% per second.

[0017] In one possible embodiment, the adaptive training module generates personalized training schemes based on historical cognitive load data and using transfer learning algorithms. It injects multi-dimensional sudden failures (such as engine failure, sudden weather changes, system alarms, and communication interruptions) coupled with high cognitive load scenarios into the stress scenario simulation system, and monitors the pilot's stress response patterns in real time through eye trackers and EEG caps, and combines machine learning algorithms to identify behavioral characteristics under stress.

[0018] Advantages and beneficial effects of the present invention: Through the aforementioned invention, this system can dynamically allocate tasks under complex weather conditions, significantly reducing the cognitive load on pilots. Verified through flight simulator and field tests, the system reduces task allocation response time to 1.2 seconds, improves task completion efficiency by 20%, reduces pilot error rate by 42% and distraction rate by 35% under high-load tasks. Furthermore, the system exhibits good stability in complex environments, with autopilot takeover adjustment accuracy reaching ±5%, effectively overcoming the limitations of existing fixed task allocation modes and achieving dynamic optimization of task allocation and intelligent upgrade of human-machine collaboration. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of the overall system architecture; Figure 2 Here is the flowchart for the load forecasting module; Figure 3 This is a schematic diagram of the task arbitration mechanism; Figure 4 This is a flowchart of the adaptive training module. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The features and illustrative embodiments of various aspects of the present invention will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific setups and methods set forth below, but covers any improvements, substitutions, and modifications to structures, methods, and devices without departing from the spirit of the invention. Well-known structures and techniques are not shown in the drawings and the following description to avoid unnecessarily obscuring the invention.

[0023] In the description of this invention, it should be noted that the directions or positional relationships indicated by terms such as "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing and simplifying the invention, and should not be construed as limiting the invention. Furthermore, the use of ordinal numbers (e.g., "first and second," etc.) is for distinguishing objects and is not limited to this order, and should not be construed as indicating or implying relative importance.

[0024] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly, encompassing both direct connection and indirect connection via an intermediate medium. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances.

[0025] It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other, and the various embodiments can be referenced and cited in each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0027] First, a pilot's physiological data is collected in real time using a wearable device integrating multiple high-precision sensors. This data includes, but is not limited to, key indicators such as heart rate variability (HRV), electroencephalogram (EEG) spectral characteristics, eye-tracking data (fixation point distribution, pupil diameter change rate, blink frequency), skin conductance response (GSR), and electromyography (EMG). Combined with historical mission performance data from the flight history database (such as emergency handling success rate, operational response time, and mission completion score) and personalized feature evaluation results (based on a comprehensive analysis of decision-making style tests, reaction speed tests, spatial cognitive ability tests, and stress tolerance tests), a pilot cognitive model is constructed. This model employs an improved LSTM-RNN algorithm, combined with a graph convolutional network (GCN) for spatiotemporal feature fusion analysis of multi-source heterogeneous data. By introducing an attention mechanism and residual connections, it predicts cognitive load trends with a 5-second time window, achieving a prediction accuracy of 92.3%. Figure 2 When the system predicts that the cognitive load will exceed a preset threshold (based on 1.5 times the standard deviation of the individual baseline value, and considering dynamic adjustments to the task environment) within the next 5-10 seconds, the intelligent task arbitration module is immediately activated. This module uses a dynamic task priority matrix algorithm, combined with a reinforcement learning mechanism, to construct a quantitative evaluation system from three dimensions: safety (based on task failure risk coefficient, system criticality index, and environmental impact factor), timeliness (considering remaining operation time margin, task urgency factor, and time pressure index), and operational complexity (comprehensively considering the number of required operation steps, interface switching frequency, and cognitive resource consumption rate). It calculates the priority weight of each task using fuzzy hierarchical analysis (FAHP) and entropy method, generates the optimal dynamic task sequence, and updates the task queue in real time.

[0028] For tasks requiring reassignment, the system transmits instructions through a multimodal interaction channel: the voice interaction system uses a natural language generation engine based on the Transformer architecture, combined with the pilot's personalized language model, to synthesize instructions that conform to the pilot's cognitive habits. At the same time, the tactile feedback device (a high-frequency vibration module integrated into the control stick) outputs tactile signals according to a preset encoding scheme, ensuring that the pilot seamlessly receives task switching information within 0.5 seconds. Figure 3 Meanwhile, the intelligent driving system employs a deep reinforcement learning-based dynamic intervention adjustment algorithm, combined with a Markov decision process (MDP), to perform gradient adjustments within a 0%-100% range based on real-time cognitive load prediction results: when the cognitive load exceeds a threshold, the system increases the autonomous driving intervention ratio at a rate of 5% per second, and restores manual operation privileges at a rate of 3% per second after detecting a decrease in load. The system also introduces a Bayesian network-based anomaly detection mechanism to monitor the matching degree between the pilot's state and system behavior in real time, ensuring smooth human-machine collaboration.

[0029] The adaptive training module generates personalized training schemes based on historical cognitive load data and uses transfer learning algorithms. It injects multi-dimensional sudden failures (such as engine failure, sudden weather changes, system alarms, and communication interruptions) coupled with high cognitive load scenarios into the stress scenario simulation system. It also monitors the pilot's stress response patterns in real time through eye trackers and EEG caps, and combines machine learning algorithms to identify behavioral characteristics under stress.

[0030] Metacognitive monitoring technology collects pilots' self-assessment data through an intelligent interactive interface (using the NASA-TLX scale for subjective workload scoring, combined with a situational awareness questionnaire), compares and analyzes the data with the system's assessment results, and uses a Bayesian optimization algorithm to dynamically adjust the training difficulty and scenario complexity, thereby achieving personalized adaptation of training content. Figure 4 In addition, the system also introduces a training data storage mechanism based on blockchain technology to ensure data security and traceability.

[0031] Example 1: System Architecture and Module Implementation like Figure 1 As shown, the aviation human-machine collaborative task allocation system based on cognitive load prediction provided in this embodiment includes the following modules: Load prediction module: Data acquisition unit: Real-time acquisition of pilot physiological data, including heart rate variability (HRV), electroencephalogram (alpha / beta wave power ratio), and eye tracking data (gaze distribution and pupil diameter) through wearable biosensors (such as EEG electrodes integrated into flight helmets and heart rate monitoring belts).

[0032] Modeling Unit: Constructs a pilot's cognitive state model, integrating flight history (mission type, emergency response score) and personality traits (decision-making style assessment results, such as risk preference index). The input dimension is physiological data × 10-second window, and the mission complexity level is (1... 5), Environmental pressure coefficient (0 1) Physiological data × 10-second window, task complexity level (1) 5), Environmental pressure coefficient (0 1).

[0033] LSTM-RNN prediction algorithm: The network structure contains 3 layers of LSTM units, with a hidden layer dimension of 64. The loss function uses mean squared error (MSE), and the output is a predicted cognitive load value for the next 5-10 seconds (normalized to 0-1). The formula is as follows:

[0034] in, For the input feature vector, In hidden state, To predict load values.

[0035] Task Arbitration Module: Dynamic Task Priority Matrix: Task priority P is calculated based on three dimensions: safety (S), timeliness (T), and operational complexity (C).

[0036] When S=1 (involving the risk of a plane crash), the task is forcibly placed on top.

[0037] Hybrid command allocation unit: Voice interaction: Utilizes directional bone conduction headphones to generate spatialized prompts (left / right channels distinguish task types), with a signal-to-noise ratio ≥20dB. Haptic feedback: The joystick vibration pattern is encoded as: high-frequency continuous vibration (emergency tasks) and low-frequency intermittent vibration (preparatory tasks).

[0038] Autonomous driving takeover algorithm: Takeover levels are divided into: Level 1: Voice prompts (0% takeover); Level 2: Roll axis control (50% takeover); Level 3: Fully automatic control (100% takeover), switching interval ≥ 2 seconds.

[0039] Adaptive Training Module: Stress Situation Simulation Unit: Injects coupled faults (such as "engine failure + instrument black screen") into the flight simulator, and simultaneously adds cognitive interference items (random arithmetic problem pop-ups, the frequency of which is positively correlated with the load threshold).

[0040] Metacognitive monitoring unit: When the difference rate between the pilot's subjective assessment and the system's objective load curve is greater than 15%, special training is triggered, and the training difficulty increases by 10% per cycle.

[0041] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.

Claims

1. An aviation human-machine collaborative task allocation system based on cognitive load prediction, characterized in that, It includes a load prediction module, a mission arbitration module, and an adaptive training module; the load prediction module establishes a pilot cognitive model and integrates multi-source heterogeneous data, including pilot physiological data, flight history data, and personalized feature data, to achieve dynamic monitoring and prediction of the pilot's cognitive state. The task arbitration module evaluates tasks based on a dynamic task priority matrix, considering safety, timeliness, and operational complexity, and achieves a smooth transition in task allocation through a multimodal interaction system. The adaptive training module generates personalized training plans based on cognitive load characteristics and optimizes training effects through a stress scenario simulation system, thereby improving pilots' ability to cope with complex environments.

2. The aviation human-machine collaborative task allocation system based on cognitive load prediction according to claim 1, characterized in that, The load prediction module uses an LSTM-RNN-based load trend prediction algorithm, combined with an attention mechanism and a graph convolutional network, to predict cognitive load trends in a rolling manner with a 5-second time window.

3. The aviation human-machine collaborative task allocation system based on cognitive load prediction according to claim 2, characterized in that, The LSTM-RNN algorithm constructs a cognitive load prediction model by collecting pilots' physiological data, flight mission data, and personality characteristic data in real time, and outputs the load change trend. The model achieves accurate prediction of cognitive load through multi-source data fusion and spatiotemporal feature extraction, and supports dynamic adjustment of prediction thresholds to adapt to different mission environments.

4. The aviation human-machine collaborative task allocation system based on cognitive load prediction according to claim 1, characterized in that, The task arbitration module calculates the priority weight of each task using fuzzy hierarchical analysis and entropy method, generates the optimal dynamic task sequence, and updates the task queue in real time.

5. The aviation human-machine collaborative task allocation system based on cognitive load prediction according to claim 1, characterized in that, The multimodal interaction system specifically refers to a hybrid command distribution system that combines voice interaction and haptic feedback; the voice interaction system uses a natural language generation engine based on the Transformer architecture to generate commands that conform to the pilot's cognitive habits; The haptic feedback device is integrated into the control stick and outputs coded haptic signals through a high-frequency vibration module, ensuring that the pilot can seamlessly receive mission switching information within 0.5 seconds.

6. The aviation human-machine collaborative task allocation system based on cognitive load prediction according to claim 1, characterized in that, The task arbitration module includes a dynamic adjustment algorithm for the autopilot system's takeover level, supporting a gradual switch from fully manual to fully automatic operation. This algorithm is based on deep reinforcement learning and Markov decision processes (MDPs). According to the pilot's real-time cognitive load and task requirements, it adjusts the autopilot system's takeover level at a rate of 5% per second. When the cognitive load exceeds a threshold, the system gradually increases the autopilot takeover ratio. When the load decreases, it restores manual operation privileges at a rate of 3% per second, achieving a dynamic balance in human-machine collaboration.

7. The aviation human-machine collaborative task allocation system based on cognitive load prediction according to claim 1, characterized in that, The adaptive training module generates personalized training plans based on cognitive load feature analysis to optimize pilots' mission execution capabilities. Through transfer learning algorithms, the system can identify pilots' weaknesses in cognitive load and design targeted training content to improve pilots' performance in high-load missions.

8. The aviation human-machine collaborative task allocation system based on cognitive load prediction according to claim 1, characterized in that, The adaptive training module has developed metacognitive monitoring technology, which collects pilots' self-assessment data through an intelligent interactive interface and compares and analyzes it with the system's assessment results. Using a Bayesian optimization algorithm, the system can dynamically optimize the training program, helping pilots better understand their own state, correct self-cognitive biases, and improve their ability to cope with complex flight missions. In addition, the training data adopts a storage mechanism based on blockchain technology to ensure data security and traceability.

9. A method for assigning human-machine collaborative tasks in aviation based on cognitive load prediction, characterized in that, The aviation human-machine collaborative task allocation system based on cognitive load prediction as described in any one of claims 1-8 includes the following process: Predict the pilot's cognitive load using the load prediction module; When it is predicted that the cognitive load will exceed the preset threshold within the next 5-10 seconds, the task arbitration module is immediately activated; The task arbitration module generates the optimal dynamic task sequence and updates the task queue in real time; Based on the priority order of each task in the task queue, instructions are transmitted to the pilot through a multimodal interaction channel to ensure that the pilot receives task switching information seamlessly within 0.5 seconds. At the same time, based on the predicted cognitive load of the pilot, a dynamic adjustment algorithm for the degree of takeover of the autopilot system is adopted to achieve a gradual switch from fully manual to fully automatic.

Images