A functional architecture and method for an intelligent human-machine adaptive collaborative system in an aircraft cockpit

Abstract

This invention provides a functional architecture and method for an intelligent human-machine adaptive collaborative system in an aircraft cockpit. It features advantages such as hierarchy, modularity, and reusability, supporting detailed design of the physical architecture of the intelligent human-machine system in the aircraft cockpit and providing a technical means to achieve efficient human-machine matching and dynamic natural interaction in the aircraft cockpit. The intelligent human-machine adaptive collaborative method provided by this invention, by adjusting the human-machine interaction level, the autonomous dimension of the intelligent agent, the information processing dimension, and the task allocation dimension, and based on the decision steps of the information flow graph, can obtain the optimal intelligent human-machine adaptive collaborative solution for the aircraft cockpit. The system functional architecture and method provided by this invention can realize intelligent human-machine interaction that adapts to the pilot's physiological, psychological, behavioral state and task requirements. It can provide technical support for solving problems such as pilot cognitive load imbalance, loss of situational awareness, and frequent human errors, enhancing the pilot's situational judgment and decision-making capabilities, and optimizing the overall effectiveness of the cockpit human-machine system.

Description

Technical Field

[0001] This invention belongs to the field of human-computer interaction technology for complex systems, and relates to a functional architecture and method for an intelligent human-computer adaptive collaborative system in an aircraft cockpit. Background Technology

[0002] As the flight mission environment develops towards networking, systematization, and clustering, trends such as multi-dimensional space, diverse forces, varied styles, and accelerated pace are becoming increasingly prominent. Driven by the development of new human-computer interaction and display control technologies, aircraft cockpits are becoming increasingly intelligent, integrated, and complex, exhibiting characteristics such as large information capacity, complex interaction processes, and high real-time requirements. This can easily lead to problems such as pilot cognitive load imbalance, loss of situational awareness, and frequent human errors.

[0003] Currently, the adaptive and collaborative capabilities of aircraft cockpit human-machine systems are insufficient, failing to meet increasingly urgent mission requirements. Key issues such as the compatibility between autonomous systems and pilot capabilities, the impact of autonomous capabilities on missions, and the overall effectiveness of human-machine systems remain unresolved, hindering the achievement of natural interaction and adaptive collaboration between humans and autonomous systems in dynamic mission environments. Furthermore, related technological research lacks comprehensiveness, exhibiting weaknesses in collaborative modes, learning capabilities, and system architecture, thus impacting the integrated application of next-generation artificial intelligence technologies in aircraft cockpit human-machine systems.

[0004] The intelligent human-machine adaptive collaboration and system functional architecture technology for aircraft cockpits, with the autonomous capabilities of intelligent agents as its core, the human-machine collaborative mode as its support, and the human-machine team formation as its foundation, possesses important characteristics such as autonomy, learning, generalization, two-way interaction, and proactive adaptability. It can achieve efficient collaborative decision-making between humans and autonomous systems, reduce the number of pilots, improve mission execution efficiency, and effectively address challenges such as situational complexity, time constraints, and environmental uncertainty. It helps improve the agility and flexibility of situational awareness, status monitoring, mission management, and tactical decision-making, and plays a significant role in establishing joint human-machine intent and common goals, enhancing situational judgment and decision-making capabilities, and optimizing the overall effectiveness of human-machine systems. Summary of the Invention

[0005] To address the shortcomings in the adaptive capabilities and collaborative levels of aircraft cockpit human-machine systems, this invention provides a functional architecture and method for an intelligent human-machine adaptive collaborative system in an aircraft cockpit. This method and system functional architecture can make adaptive decisions based on data such as the pilot's cognitive state and the autonomous capabilities of the intelligent agent. It possesses characteristics such as universality, reusability, and portability, and can optimize the overall performance of aircraft cockpit human-machine systems.

[0006] The first aspect of the present invention provides an intelligent human-machine adaptive collaborative system for an aircraft cockpit, comprising: an intelligent human-machine module group, an intelligent agent perception system module, an intelligent agent decision-making system module, and an intelligent agent execution system module;

[0007] The intelligent human-machine module group includes: a display module, a status monitoring module, an adaptive collaboration module, and a control module. The functions of each module are connected in the logical order of perception, decision-making, and execution. The input data comes from the pilot's decision-making and execution behavior as well as the perception, decision-making, and execution system modules of the intelligent agent. The output data affects the pilot's perception behavior.

[0008] The intelligent agent perception system module receives data from the flight mission environment, aircraft dynamics system, and intelligent agent execution system module. This module acquires mission situation, aircraft status, and intelligent agent execution results, which are presented to the pilot through the display module of the intelligent human-machine module group, and provide basic input and decision-making basis for the intelligent agent decision-making system module.

[0009] The intelligent agent decision-making system module receives data from the intelligent agent perception system module. After rule matching and knowledge reasoning, the module presents the decision results to the pilot through the display module of the intelligent human-machine module group. On the other hand, it provides a basis for the deep processing of the status monitoring module, adaptive collaboration module, and intelligent agent execution system module.

[0010] The intelligent agent execution system module receives data from the intelligent agent perception and decision-making system module, and is used to realize human-machine team collaboration through the adaptive collaboration module of the intelligent human-machine module group, and / or to directly control the aircraft body dynamics system through the control module of the intelligent human-machine module group. The execution result of this module will be returned to the intelligent agent perception system module.

[0011] The state monitoring module of the intelligent human-machine module group includes a data acquisition terminal layer, a data acquisition layer, a feature extraction layer, a feature fusion layer, and an external interface layer. The data sources for state monitoring include physiological state data and behavioral state data. Among them, physiological features include physiological state parameters such as pilot eye movement, electroencephalogram, electrocardiogram, electromyography, and electrodermal conductance, while behavioral features include behavioral state parameters of the pilot and the intelligent agent during human-machine interaction.

[0012] The adaptive collaboration module of the intelligent human-machine module group uses the state characteristics of the pilot and the intelligent agent as the basis for decision-making. Based on the information flow graph decision method, it obtains the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit, realizing the adaptive adjustment of macroscopic human-machine interaction level and microscopic variable degrees of freedom.

[0013] Optionally, the adaptive collaboration module is used to, based on the pilot's physiological, psychological, and behavioral states as triggering conditions, macroscopically switch the aircraft cockpit human-machine interaction level between fully human-controlled interaction level L1, human-host-assisted interaction level L2, human-machine negotiation interaction level L3, pilot-host-assisted interaction level L4, and fully autonomous interaction level L5, and microscopically adjust the relative ranges of the agent's autonomy dimension D1, information processing dimension D2, and task allocation dimension D3 according to at least one of the following principles: priority of information processing process, priority of agent's autonomous capability, and minimum risk level, thereby establishing an aircraft cockpit intelligent human-machine adaptive collaboration strategy; wherein,

[0014] The physiological, psychological, and behavioral state of pilots includes cognitive load, situational awareness, and human error.

[0015] Optionally, the adaptive coordination module is used to perform the following steps:

[0016] 1) Determine the set of feasible adaptive cooperative adjustment schemes S = {s1, s2, ..., s} n}, where n represents the total number of feasible solutions, and each element in the set S represents a feasible solution;

[0017] 2) Determine the set of performance evaluation indicators F = {f1, f2, ..., f...} for each feasible adaptive collaborative adjustment scheme. m}, where m represents the total number of all influencing factors, and each element in set F represents an influencing factor;

[0018] 3) Under the measure of the performance evaluation index set F, compare all feasible solutions in set S pairwise to determine the score set R = {r1, r2, ..., r...} for each comparison. m}, weight set Ω={ω1,ω2,…,ω m In each set of scores and weights, there is a one-to-one correspondence between the values ​​and elements of the evaluation index set, with "+" and "-" symbols representing positive and negative correlations, respectively. The results of pairwise comparisons are then used. ij The result was obtained through linear weighting:

[0019] l ij =ω1r1+ω2r2+Lω m r m

[0020] Where i and j represent two feasible solutions in set S, 1≤i≤n, 1≤j≤n, l ij If l is positive, it means that solution i is better than solution j. ij A negative number indicates that solution i is inferior to solution j. ij A value of 0 indicates that scheme i and scheme j are of equal merit.

[0021] 4) Treat each feasible solution as an independent node, and connect each pair of solutions with an edge to form a complete information flow graph, using l ij The absolute value of the value represents the information flow load of each edge, expressed in l. ij The "+" and "-" signs determine the direction of the edge, and the arrow points in the "+" direction. ij If the value is 0, the edge can point to any direction.

[0022] 5) Calculate the information flow load of node i in the complete information flow graph.

[0023] 6) Sort all nodes in the complete information flow graph according to the information flow load. The node with the larger information flow load has a higher priority for the corresponding feasible solution. This can determine the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0024] Optionally, the adaptive coordination module is used to control the agent's autonomy dimension D1 to remain unchanged, while adjusting the information processing dimension D3 and the task allocation dimension D2, under the priority principle of information processing.

[0025] The information processing priority principle is to prioritize the information processing process of information acquisition, information analysis, judgment and decision-making, and action implementation in the task, and appropriately increase the information processing responsibilities of the pilot. When adjusting the scope of responsibilities of the pilot / intelligent agent in the information processing dimension alone is no longer sufficient to meet the needs of task execution, the human-computer interaction level is reduced, and more or more difficult tasks are assigned to the pilot for processing.

[0026] Optionally, the adaptive coordination module is used to adjust the agent's autonomy dimension D1 and task allocation dimension D3 while keeping the control information processing dimension D2 unchanged, under the principle of minimizing the degree of danger.

[0027] The principle of minimizing the degree of danger means that for emergencies with a lower degree of danger, the autonomy dimension of the intelligent agent is adjusted first to keep the human-machine system at the original level of human-machine interaction. For emergencies with a higher degree of danger, the task allocation between the pilot and the intelligent agent is adjusted by reducing the level of human-machine interaction to increase the pilot's task participation.

[0028] Optionally, the adaptive coordination module is used to adjust the agent's autonomy dimension D1 and information processing dimension D2 while keeping the task allocation dimension D3 unchanged, under the principle of prioritizing the agent's autonomy.

[0029] The principle of prioritizing the autonomous capabilities of the intelligent agent is to prioritize adjusting the autonomous dimension of the intelligent agent to adapt to the requirements of the current task allocation status and information processing responsibilities. When adjusting the autonomous dimension of the intelligent agent alone is no longer sufficient to meet the needs of the current task execution, the information processing responsibilities of the pilot and the intelligent agent are adjusted by changing the level of human-computer interaction.

[0030] Optionally, the data acquisition terminal layer of the intelligent human-machine module group status monitoring module includes terminal devices for monitoring the pilot's physiological state, such as eye-tracking cameras, EEG amplifiers, ECG sensors, EMG sensors, and EEG sensors, as well as terminal devices for monitoring the pilot's and intelligent agent's behavioral state, such as motion capture devices and operation logs.

[0031] The data acquisition layer of the intelligent human-machine module group status monitoring module processes the raw data collected by the terminal through the controller and sends out structured, time-synchronized multimodal physiological, action data and instruction data.

[0032] The feature extraction layer of the intelligent human-machine module group status monitoring module extracts the corresponding physiological features, action features and instruction semantics.

[0033] The feature fusion layer of the intelligent human-machine module group state monitoring module realizes the fusion of physiological state features and behavioral state features, and further fusion of the two types of features by combining the context of system state and task situation.

[0034] The external interface layer of the intelligent human-machine module group status monitoring module provides external application interfaces for cognitive state assessment, interaction intent recognition, behavioral performance assessment, and human error recognition, providing decision-making basis for intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0035] A second aspect of the present invention provides an intelligent human-machine adaptive collaborative method for aircraft cockpits, which is implemented using the system described in any one of the first aspects.

[0036] This invention provides a functional architecture and method for an intelligent human-machine adaptive collaborative system in an aircraft cockpit. It features advantages such as hierarchy, modularity, and reusability, supporting detailed design of the physical architecture of the intelligent human-machine system in the aircraft cockpit and providing a technical means to achieve efficient human-machine matching and dynamic natural interaction in the aircraft cockpit. The intelligent human-machine adaptive collaborative method provided by this invention, by adjusting the human-machine interaction level, the autonomous dimension of the intelligent agent, the information processing dimension, and the task allocation dimension, and based on the decision steps of the information flow graph, can obtain the optimal intelligent human-machine adaptive collaborative solution for the aircraft cockpit. The system functional architecture and method provided by this invention can realize intelligent human-machine interaction that adapts to the pilot's physiological, psychological, behavioral state and task requirements. It can provide technical support for solving problems such as pilot cognitive load imbalance, loss of situational awareness, and frequent human errors, enhancing the pilot's situational judgment and decision-making capabilities, and optimizing the overall effectiveness of the cockpit human-machine system. Attached Figure Description

[0037] Figure 1 A diagram illustrating the overall architecture of an intelligent human-machine adaptive collaborative system for aircraft cockpits.

[0038] Figure 2 This is a functional architecture diagram of the intelligent human-machine module group status monitoring module;

[0039] Figure 3 A human-machine adaptive collaboration strategy that keeps the agent's autonomy dimension (D1) constant;

[0040] Figure 4 A human-machine adaptive collaboration strategy that keeps the information processing dimension (D2) constant;

[0041] Figure 5 A human-machine adaptive collaboration strategy is proposed to keep the task allocation dimension (D3) constant. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] The features and illustrative embodiments of various aspects of the present invention will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific setups and methods set forth below, but covers any improvements, substitutions, and modifications to structures, methods, and devices without departing from the spirit of the invention. Well-known structures and techniques are not shown in the drawings and the following description to avoid unnecessarily obscuring the invention.

[0044] It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other, and the various embodiments can be referenced and cited in each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0046] This invention provides a functional architecture for an intelligent human-machine adaptive collaborative system in an aircraft cockpit. The system functional architecture includes an intelligent human-machine module group, an intelligent agent perception system module, an intelligent agent decision-making system module, and an intelligent agent execution system module.

[0047] The intelligent human-machine module group includes a display module, a status monitoring module, an adaptive collaboration module, and a control module. The functions of each module are connected in the logical order of perception, decision-making, and execution. The input data comes from the pilot's decision-making and execution behavior as well as the perception, decision-making, and execution system modules of the intelligent agent. The output data affects the pilot's perception behavior.

[0048] The data sources for the aforementioned status monitoring include physiological and behavioral status data. Physiological features include parameters such as pilot eye movements, electroencephalograms (EEGs), electrocardiograms (ECGs), electromyograms (EMGs), and electrodermal conductance (EDCs). Behavioral features include behavioral parameters of the pilot and the intelligent agent during human-computer interaction. The status monitoring module includes a data acquisition terminal layer, a data acquisition layer, a feature extraction layer, a feature fusion layer, and an external interface layer, such as... Figure 2 As shown, where:

[0049] (1) The data acquisition terminal layer includes terminal devices such as eye-tracking cameras, EEG amplifiers, ECG sensors, EMG sensors, and EEG sensors that monitor the physiological state of pilots, as well as terminal devices such as motion capture devices and operation logs that monitor the behavioral state of pilots and intelligent agents.

[0050] (2) The data acquisition layer processes the raw data collected by the terminal through the controller and sends out structured, time-synchronized multimodal physiological, action data and instruction data;

[0051] (3) The feature extraction layer extracts the corresponding physiological features, action features and instruction semantics;

[0052] (4) The feature fusion layer realizes the fusion of physiological state features and behavioral state features, and at the same time, combined with the context of system state and task situation, it realizes the fusion of the two types of features at a higher level.

[0053] (5) The external interface layer provides external application interfaces for cognitive state assessment, interaction intent recognition, behavioral performance assessment, and human error recognition, providing decision-making basis for intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0054] The aforementioned adaptive collaboration module uses the state characteristics of the pilot and the intelligent agent as the basis for decision-making. Based on the information flow graph decision-making method, it obtains the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit, realizing the adaptive adjustment of macroscopic human-machine interaction level and microscopic variable degrees of freedom.

[0055] The intelligent agent perception system module receives data from the flight mission environment, aircraft dynamics system, and intelligent agent execution system module. This module acquires information such as mission situation, aircraft status, and intelligent agent execution results. On the one hand, it presents the information to the pilot through the display module of the intelligent human-machine module group, and on the other hand, it provides basic input and decision-making basis for the intelligent agent decision-making system module.

[0056] The intelligent agent decision-making system module receives data from the intelligent agent perception system module. After steps such as rule matching and knowledge reasoning, the module presents the decision results to the pilot through the display module of the intelligent human-machine module group. On the other hand, it provides a basis for the deep processing of the status monitoring module, adaptive collaboration module, and intelligent agent execution system module.

[0057] The intelligent agent execution system module receives data from the intelligent agent perception and decision-making system module. This module can achieve human-machine team collaboration through the adaptive collaboration module of the intelligent human-machine module group, or it can directly control the aircraft's dynamics system through the control module of the intelligent human-machine module group. The execution result of this module will be returned to the intelligent agent perception system module.

[0058] The state monitoring module of the intelligent human-machine module group includes a data acquisition terminal layer, a data acquisition layer, a feature extraction layer, a feature fusion layer, and an external interface layer. The data sources for state monitoring include physiological state data and behavioral state data. Among them, physiological features include physiological state parameters such as pilot eye movement, electroencephalogram, electrocardiogram, electromyography, and electrodermal conductance, while behavioral features include behavioral state parameters of the pilot and the intelligent agent during human-machine interaction.

[0059] This invention provides an intelligent human-machine adaptive collaboration method for aircraft cockpits, comprising the following steps:

[0060] (1) The intelligent human-machine adaptive collaborative system functional architecture of the aircraft cockpit is adopted, which includes an intelligent human-machine module group, an intelligent agent perception system module, an intelligent agent decision-making system module, and an intelligent agent execution system module.

[0061] (2) Based on the characteristics of human-computer interaction tasks, define discrete aircraft cockpit human-computer interaction levels from a macro perspective and determine the overall boundary of human-computer collaboration mode;

[0062] The aircraft cockpit human-machine interaction levels include: L1, L2, L3, L4, and L5. The human-machine collaboration modes include: fully human-controlled, human-machine-assisted, human-machine-negotiated, pilot-machine-assisted, and fully autonomous. The L1 interaction level corresponds to the "fully human-controlled" collaboration mode, the L2 interaction level corresponds to the "human-machine-assisted" collaboration mode, the L3 interaction level corresponds to the "human-machine-negotiated" collaboration mode, the L4 interaction level corresponds to the "pilot-machine-assisted" collaboration mode, and the L5 interaction level corresponds to the "fully autonomous" collaboration mode.

[0063] (3) Based on the level of the agent's autonomy, determine the internal attributes of the human-machine collaboration mode from a microscopic perspective according to the control direction of different variable degrees of freedom;

[0064] The variable degrees of freedom include: agent autonomy dimension (D1), information processing dimension (D2), and task allocation dimension (D3);

[0065] (4) Using the pilot’s cognitive load, situational awareness, human error and other physiological, psychological and behavioral states as triggering conditions, the aircraft cockpit human-machine interaction level is switched between L1 and L5 on a macro level, and the relative range of the intelligent agent’s autonomous dimension (D1), information processing dimension (D2) and task allocation dimension (D3) is adjusted on a micro level to establish an intelligent human-machine adaptive collaborative strategy for the aircraft cockpit.

[0066] Specifically, the adaptive strategy for the agent's autonomy dimension involves adjusting the boundaries of what the agent "has the ability to do" and "is allowed to do" from both capability and constraint perspectives; the adaptive strategy for the information processing dimension corresponds to the "Observe-Adjust-Decision-Action (OODA)" loop, adjusting the allocation of responsibilities between the pilot and the agent in the vertical information processing stages such as information acquisition, information analysis, judgment and decision-making, and action implementation; and the adaptive strategy for the task allocation dimension adjusts the task allocation and decision-making authority between the human and machine based on the task environment of the cockpit human-machine system, the pilot's workload, and system performance.

[0067] (5) Keep the autonomous dimension (D1) of the control agent unchanged, and adjust the information processing dimension (D3) and task allocation dimension (D2) according to the priority principle of information processing process;

[0068] The information processing priority principle is to prioritize the information processing processes of task acquisition, information analysis, judgment and decision-making, and action implementation, and appropriately increase the information processing responsibilities of the pilot. When simply adjusting the scope of responsibilities of the pilot / intelligent agent in the information processing dimension is no longer sufficient to meet the needs of task execution, the human-computer interaction level is reduced, and more or more difficult tasks are assigned to the pilot for processing.

[0069] (6) Keeping the control information processing dimension (D2) unchanged, and following the principle of minimizing the degree of danger, adjust the agent autonomy dimension (D1) and task allocation dimension (D3);

[0070] The principle of minimizing the degree of danger means that for emergencies with a lower degree of danger, the autonomous dimension of the intelligent agent is adjusted first to keep the human-machine system at the original level of human-machine interaction. For emergencies with a higher degree of danger, the task allocation between the pilot and the intelligent agent is adjusted by reducing the level of human-machine interaction to increase the pilot's task participation.

[0071] (7) Keep the task allocation dimension (D3) unchanged, and follow the principle of prioritizing the agent's autonomous capabilities, adjust the agent's autonomous dimension (D1) and information processing dimension (D2);

[0072] The principle of prioritizing the autonomous capabilities of the intelligent agent is to prioritize adjusting the autonomous dimension of the intelligent agent to adapt to the requirements of the current task allocation status and information processing responsibilities. When adjusting the autonomous dimension of the intelligent agent alone is no longer sufficient to meet the needs of the current task execution, the information processing responsibilities of the pilot and the intelligent agent are adjusted by changing the level of human-computer interaction.

[0073] (8) Based on the information flow graph decision method, the optimal intelligent human-machine adaptive collaborative solution for the aircraft cockpit is obtained for all feasible macro-level and micro-dimensional combination adjustment schemes.

[0074] Furthermore, the specific steps for obtaining the optimal intelligent human-machine adaptive collaborative solution for the aircraft cockpit based on the information flow graph decision method are as follows:

[0075] 1) Determine the set of feasible adjustment schemes S = {s1, s2, ..., s} n}, where n represents the total number of feasible solutions, and each element in the set S represents a feasible solution;

[0076] 2) Determine the set of comprehensive performance evaluation indicators for human-machine systems, F = {f1, f2, ..., f...} m}, where m represents the total number of all influencing factors, and each element in set F represents an influencing factor;

[0077] 3) Under the measure of the evaluation index set F, compare all feasible solutions in the pairwise set S to determine the score set R = {r1, r2, ..., r...} for each comparison. m}, weight set Ω={ω1,ω2,…,ω m In each set of scores and weights, there is a one-to-one correspondence between the values ​​and elements of the evaluation index set, with "+" and "-" symbols representing positive and negative correlations, respectively. The results of pairwise comparisons are then used. ij The result was obtained through linear weighting:

[0078] l ij =ω1r1+ω2r2+Lω m r m

[0079] Where i and j represent two feasible solutions in set S (1≤i≤n, 1≤j≤n), l ij If l is positive, it means that solution i is better than solution j. ij A negative number indicates that solution i is inferior to solution j. ij A value of 0 indicates that scheme i and scheme j are of equal merit.

[0080] 4) Treat each feasible solution as an independent node, and connect each pair of solutions with an edge to form a complete information flow graph, using l ij The absolute value of the value represents the information flow load of each edge, expressed in l. ij The "+" and "-" signs determine the direction of the edge (the arrow points in the "+" direction), l ij If the value is 0, the edge can point to any direction.

[0081] 5) Calculate the information flow load of node i in the complete information flow graph.

[0082] 6) Sort all nodes in the complete information flow graph according to the information flow load. The node with the larger information flow load has a higher priority for the corresponding feasible solution. This can determine the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0083] The above description is merely a further embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope disclosed in the present invention, based on the technical solution and concept of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. An intelligent human-machine adaptive collaborative system for aircraft cockpits, characterized in that, include: Intelligent human-machine module group, intelligent agent perception system module, intelligent agent decision-making system module, and intelligent agent execution system module; The intelligent human-machine module group includes: a display module, a status monitoring module, an adaptive collaboration module, and a control module. The functions of each module are connected in the logical order of perception, decision-making, and execution. The input data comes from the pilot's decision-making and execution behavior as well as the perception, decision-making, and execution system modules of the intelligent agent. The output data affects the pilot's perception behavior. The intelligent agent perception system module receives data from the flight mission environment, aircraft dynamics system, and intelligent agent execution system module. The intelligent agent perception system acquires mission situation, aircraft status, and intelligent agent execution results, which are presented to the pilot through the display module of the intelligent human-machine module group, and provide basic input and decision-making basis for the intelligent agent decision-making system module. The intelligent agent decision-making system module receives data from the intelligent agent perception system module. After rule matching and knowledge reasoning, the intelligent agent decision-making system module presents the decision results to the pilot through the display module of the intelligent human-machine module group. On the other hand, it provides a basis for the in-depth processing of the status monitoring module, adaptive collaboration module, and intelligent agent execution system module. The intelligent agent execution system module receives data from the intelligent agent perception and decision-making system module, and is used to realize human-machine team collaboration through the adaptive collaboration module of the intelligent human-machine module group, and / or to directly control the aircraft's dynamics system through the control module of the intelligent human-machine module group. The execution result of the intelligent agent execution system module will be returned to the intelligent agent perception system module. The status monitoring module of the intelligent human-machine module group includes a data acquisition terminal layer, a data acquisition layer, a feature extraction layer, a feature fusion layer, and an external interface layer. The data sources for status monitoring include physiological features and behavioral features. Among them, physiological features include physiological state parameters of pilot eye movement, electroencephalogram, electrocardiogram, electromyography, and electrodermal conductance, and behavioral features include behavioral state parameters of pilot and intelligent agent during human-machine interaction. The adaptive collaboration module of the intelligent human-machine module group uses the state characteristics of the pilot and the intelligent agent as the basis for decision-making. Based on the information flow graph decision method, it obtains the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit, realizing the adaptive adjustment of macroscopic human-machine interaction level and microscopic variable degrees of freedom.

2. The intelligent human-machine adaptive collaborative system for aircraft cockpits according to claim 1, characterized in that, The adaptive collaboration module is used to, based on the pilot's physiological, psychological, and behavioral states as triggering conditions, macroscopically switch the aircraft cockpit human-machine interaction level between fully human-controlled interaction level L1, human-host-assisted interaction level L2, human-machine negotiation interaction level L3, pilot-host-assisted interaction level L4, and fully autonomous interaction level L5. Microscopically, it adjusts the relative ranges of the agent's autonomy dimension D1, information processing dimension D2, and task allocation dimension D3 according to at least one of the following principles: priority of information processing, priority of agent autonomy, and minimum risk. This establishes an adaptive collaboration strategy for intelligent human-machine interaction in the aircraft cockpit. The physiological, psychological, and behavioral state of pilots includes cognitive load, situational awareness, and human error.

3. The intelligent human-machine adaptive collaborative system for aircraft cockpits according to claim 2, characterized in that, The adaptive coordination module is used to perform the following steps: 1) Determine the set of feasible adaptive cooperative adjustment schemes S={ s 1, s 2, …, s n },in, n S represents the total number of all feasible solutions, and each element in the set S represents a feasible solution. 2) Determine the set of performance evaluation indicators F={ for each feasible adaptive collaborative adjustment scheme. f 1, f 2, …, f m },in, m The set F represents the total number of all influencing factors, and each element in the set F represents one influencing factor. 3) Under the measure of the performance evaluation index set F, compare all feasible solutions in set S pairwise to determine the score set R = { r 1, r 2, …, r m }, weight set Ω={ ω 1, ω 2, …, ω m In each set of scores and weights, there is a one-to-one correspondence between the values ​​and elements of the evaluation index set. "+" and "-" signs represent positive and negative correlations, respectively, and the results of pairwise comparisons are then used. l ij The result was obtained through linear weighting: in, i , j Let S represent two feasible solutions in set S, where 1 ≤ i ≤ n , 1≤ j ≤ n , l ij If it is a positive number, then the solution is explained. i Superior Solution j , l ij If it is negative, it indicates a solution. i Inferior solution j , l ij A value of 0 indicates a solution. i and plan j Equally superior or inferior; 4) Treat each feasible solution as an independent node, and connect each pair of solutions with an edge to form a complete information flow graph, so as to... l ij The absolute value of the value represents the information flow load of each edge. l ij The "+" and "-" signs determine the direction of the edge, and the arrow points in the "+" direction. l ij If the value is 0, the edge can point to any direction. 5) Calculate the nodes in the complete information flow graph. i Information flow load ; 6) Sort all nodes in the complete information flow graph according to the information flow load. The node with the larger information flow load has a higher priority for the corresponding feasible solution. This can determine the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit.

4. The intelligent human-machine adaptive collaborative system for aircraft cockpits according to claim 2, characterized in that, The adaptive coordination module is used to keep the agent's autonomy dimension D1 unchanged and adjust the information processing dimension D3 and the task allocation dimension D2 under the priority principle of information processing. The information processing priority principle is to prioritize the information processing process of information acquisition, information analysis, judgment and decision-making, and action implementation in the task, and appropriately increase the information processing responsibilities of the pilot. When adjusting the scope of responsibilities of the pilot / intelligent agent in the information processing dimension alone is no longer sufficient to meet the needs of task execution, the human-computer interaction level is reduced, and more or more difficult tasks are assigned to the pilot for processing.

5. The intelligent human-machine adaptive collaborative system for aircraft cockpits according to claim 2, characterized in that, The adaptive coordination module is used to adjust the agent's autonomy dimension D1 and task allocation dimension D3 while keeping the control information processing dimension D2 unchanged, under the principle of minimizing the degree of danger. The principle of minimizing the degree of danger means that for emergencies with a lower degree of danger, the autonomy dimension of the intelligent agent is adjusted first to keep the human-machine system at the original level of human-machine interaction. For emergencies with a higher degree of danger, the task allocation between the pilot and the intelligent agent is adjusted by reducing the level of human-machine interaction to increase the pilot's task participation.

6. The intelligent human-machine adaptive collaborative system for aircraft cockpits according to claim 2, characterized in that, The adaptive coordination module is used to adjust the agent's autonomy dimension D1 and information processing dimension D2 while keeping the task allocation dimension D3 unchanged, under the principle of prioritizing the agent's autonomy. The principle of prioritizing the autonomous capabilities of the intelligent agent is to prioritize adjusting the autonomous dimension of the intelligent agent to adapt to the requirements of the current task allocation status and information processing responsibilities. When adjusting the autonomous dimension of the intelligent agent alone is no longer sufficient to meet the needs of the current task execution, the information processing responsibilities of the pilot and the intelligent agent are adjusted by changing the level of human-computer interaction.

7. The intelligent human-machine adaptive collaborative system for aircraft cockpits according to claim 1, characterized in that, The data acquisition terminal layer of the intelligent human-machine module group status monitoring module includes terminal devices for monitoring the pilot's physiological state, such as eye-tracking cameras, EEG amplifiers, ECG sensors, EMG sensors, and EEG sensors, as well as terminal devices for monitoring the pilot's and intelligent agent's behavioral state, such as motion capture devices and operation logs. The data acquisition layer of the intelligent human-machine module group status monitoring module processes the raw data collected by the terminal through the controller and sends out structured, time-synchronized multimodal physiological, action data and instruction data. The feature extraction layer of the intelligent human-machine module group status monitoring module extracts the corresponding physiological features, action features and instruction semantics. The feature fusion layer of the intelligent human-machine module group state monitoring module realizes the fusion of physiological state features and behavioral state features, and further fusion of the two types of features by combining the context of system state and task situation. The external interface layer of the intelligent human-machine module group status monitoring module provides external application interfaces for cognitive state assessment, interaction intent recognition, behavioral performance assessment, and human error recognition, providing decision-making basis for intelligent human-machine adaptive collaboration in the aircraft cockpit.

8. A method for intelligent human-machine adaptive collaboration in an aircraft cockpit, characterized in that, The system described in any one of claims 1-7 shall be used to perform this operation.

Images