Functional architecture and method for intelligent man-machine self-adaptive cooperative system of aircraft cockpit

Abstract

The invention provides a functional architecture and method for an intelligent man-machine adaptive cooperative system of an aircraft cockpit, which have the advantages of layering, modularization, reusability and the like, can support the detailed design of a physical architecture of the intelligent man-machine system of the aircraft cockpit, and can provide a technical implementation way for efficient man-machine matching and dynamic natural interaction of the aircraft cockpit. According to the intelligent man-machine self-adaptive cooperation method for the aircraft cockpit, the optimal intelligent man-machine self-adaptive cooperation solution of the aircraft cockpit can be obtained by adjusting the man-machine interaction level, the agent autonomous dimension, the information processing dimension and the task allocation dimension and based on the information flow graph decision-making step. According to the system function architecture and the method provided by the invention, intelligent man-machine interaction self-adaptive to physiological, psychological and behavior states and task requirements of the pilot can be realized, and the problems of cognitive load imbalance, scene awareness loss, frequent human errors and the like of the pilot can be solved; the pilot situation judgment and decision-making capability is enhanced, and technical support is provided for optimizing the comprehensive efficiency of a cabin man-machine system.

Description

Technical Field

[0001] The present invention belongs to the technical field of complex system human-computer interaction, and relates to a functional architecture and method of an intelligent human-computer adaptive collaborative system in an aircraft cockpit. Background Art

[0002] As the flight mission environment develops towards networking, systematization and clustering, trends such as multi-dimensional space, diversified forces, diverse styles and accelerated pace are gradually prominent. Driven by the development of new human-computer interaction and display control technologies, the intelligence, integration and complexity of aircraft cockpits are becoming increasingly higher, showing characteristics such as large information capacity, complex interaction process and high immediacy requirements, which can easily lead to problems such as imbalance of pilot cognitive load, loss of situational awareness and frequent human errors.

[0003] At present, the adaptive ability and coordination level of the human-machine system in the aircraft cockpit are insufficient to meet the increasingly urgent mission requirements. Key issues such as the matching of autonomous systems and pilot capabilities, the impact of autonomous capabilities on missions, and the comprehensive effectiveness of human-machine systems have not been resolved. It is difficult to achieve natural interaction and adaptive coordination between humans and autonomous systems in a dynamic mission environment. Related technical research lacks integrity and has weaknesses in coordination modes, learning capabilities, system architecture, etc., which affects the comprehensive application of the new generation of artificial intelligence technology in the human-machine system in the aircraft cockpit.

[0004] The aircraft cockpit intelligent human-machine adaptive collaboration and system functional architecture technology takes the autonomous capability of intelligent agents as its core, is supported by the human-machine collaborative mode, and relies on the form of human-machine teaming. It has important characteristics such as autonomy, learning, generalization, two-way interaction, and active adaptability. It can achieve efficient collaborative decision-making between humans and autonomous systems, reduce the number of pilots, and improve mission execution efficiency. It can effectively respond to challenges such as situation complexity, time urgency, and environmental uncertainty. It helps to improve the agility and flexibility of situation recognition, status monitoring, mission management, and tactical decision-making. It plays an important role in establishing joint human-machine intentions and common goals, enhancing situation judgment and decision-making capabilities, and optimizing the comprehensive effectiveness of human-machine systems. Summary of the invention

[0005] In response to the problem of insufficient adaptability and coordination level of aircraft cockpit human-machine systems, the present invention provides a functional architecture and method for an aircraft cockpit intelligent human-machine adaptive coordination system. This method and system functional architecture can make adaptive decisions based on data such as the pilot's cognitive state and the autonomous ability of the intelligent agent. It has the characteristics of universality, reusability, and portability, and can optimize the comprehensive performance of the aircraft cockpit human-machine system.

[0006] The first aspect of the present invention provides an aircraft cockpit intelligent human-machine adaptive collaborative system, comprising: an intelligent human-machine module group, an intelligent agent perception system module, an intelligent agent decision system module, and an intelligent agent execution system module;

[0007] The intelligent human-machine module group includes: a display module, a state monitoring module, an adaptive collaboration module, and a control module. The functions of each module are connected in the logical order of perception, decision-making, and execution. The access data comes from the pilot's decision-making and execution behavior and the perception, decision-making, and execution system modules of the intelligent body. The output data affects the pilot's perception behavior.

[0008] The intelligent agent perception system module accesses data from the flight mission environment, the aircraft body dynamics system, and the intelligent agent execution system module. The module obtains the mission situation, body status, and intelligent agent execution results, and presents them to the pilot through the display module of the intelligent human-machine module group on the one hand, and provides basic input and decision-making basis for the intelligent agent decision system module on the other hand;

[0009] The intelligent agent decision system module accesses the data from the intelligent agent perception system module. After rule matching and knowledge reasoning, the module presents the decision results to the pilot through the display module of the intelligent human-machine module group on the one hand, and provides a basis for the deep processing of the state monitoring module, the adaptive collaboration module, and the intelligent agent execution system module on the other hand;

[0010] The intelligent agent execution system module accesses data from the intelligent agent perception and decision-making system module, and is used to realize human-machine team collaboration through the adaptive collaboration module of the intelligent human-machine module group, and / or directly control the aircraft body dynamics system through the control module of the intelligent human-machine module group. The execution result of this module will be returned to the intelligent agent perception system module;

[0011] The state monitoring module of the intelligent human-machine module group includes an acquisition terminal layer, a data acquisition layer, a feature extraction layer, a feature fusion layer, and an external interface layer. The data sources of the state monitoring include physiological state data and behavioral state data, wherein the physiological characteristics include physiological state parameters such as eye movement, electroencephalogram, electrocardiogram, electromyography, and skin electricity of the pilot, and the behavioral characteristics include behavioral state parameters of the pilot and the intelligent agent during the human-machine interaction process;

[0012] The adaptive collaboration module of the intelligent human-machine module group takes the state characteristics of the pilot and the intelligent body as the decision-making basis, obtains the optimal solution for the intelligent human-machine adaptive collaboration in the aircraft cockpit based on the information flow diagram decision-making method, and realizes the adaptive adjustment of the macro human-machine interaction level and the micro variable degree of freedom.

[0013] Optionally, the adaptive collaboration module is used to switch the aircraft cockpit human-machine interaction level between the full human-control interaction level L1, the human-host-assistant interaction level L2, the human-machine negotiation interaction level L3, the aircraft-host-assistant interaction level L4, and the fully autonomous interaction level L5 based on the pilot's physiological, psychological, and behavioral states as trigger conditions. At the macro level, it is used to adjust the relative range of the intelligent agent autonomy dimension D1, the information processing dimension D2, and the task allocation dimension D3 according to at least one of the information processing process priority principle, the intelligent agent autonomy capability priority principle, and the minimum danger level principle, and establish an aircraft cockpit intelligent human-machine adaptive collaboration strategy; wherein,

[0014] The pilot's physiological, psychological, and behavioral states include pilot cognitive load, situational awareness, and human errors.

[0015] Optionally, the adaptive collaboration module is used to perform the following steps:

[0016] 1) Determine a feasible adaptive collaborative adjustment solution set S = {s 1 ,s 2 ,…,s n}, where n represents the total number of feasible solutions, and each element in the set S represents a feasible solution;

[0017] 2) Determine the performance evaluation index set F = {f 1 ,f 2 ,…,f m}, where m represents the total number of influencing factors, and each element in the set F represents an influencing factor;

[0018] 3) Under the measurement of the performance evaluation index set F, compare all feasible solutions in the set S pairwise and determine the score set R = {r 1 ,r 2 ,…,r m}, weight set Ω = {ω 1 ,ω 2 ,…,ω m}, each value in the score set and weight set corresponds to the element of the evaluation index set, and the "+" and "-" symbols are used to indicate positive correlation and negative correlation respectively. The results of pairwise comparison are l ij The linear weighting method yields:

[0019] l ij =ω 1 r 1 +ω 2 r 2 +Lω m r m

[0020] Among them, i and j represent two feasible solutions in the set S, 1≤i≤n, 1≤j≤n, l ij If it is a positive number, it means that solution i is better than solution j. ij If it is a negative number, it means that plan i is worse than plan j. ij If it is 0, it means that scheme i and scheme j are equally good;

[0021] 4) Treat each feasible solution as an independent node, and connect each solution with an edge to form a complete information flow graph. ij The absolute value of represents the information flow load of each edge, expressed as l ij The "+" and "-" signs determine the direction of the edge, and the arrow points to the "+" direction. ij If it is 0, the edge can point to any direction;

[0022] 5) Calculate the information flow load of node i in the complete information flow graph

[0023] 6) Sort all nodes in the complete information flow graph according to the size of the information flow load. The node with a larger information flow load has a higher priority for the corresponding feasible solution, thereby determining the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0024] Optionally, the adaptive collaboration module is used to control the agent autonomy dimension D1 to remain unchanged and adjust the information processing dimension D3 and the task allocation dimension D2 under the principle of information processing process priority;

[0025] Among them, the information processing process priority principle is to prioritize the information acquisition, information analysis, judgment and decision-making, and action implementation of the task, appropriately increase the pilot's information processing responsibilities, and when only adjusting the scope of pilot / agent responsibilities in the information processing dimension can no longer meet the needs of task execution, by reducing the level of human-computer interaction, more or more difficult tasks will be assigned to the pilot for processing.

[0026] Optionally, the adaptive collaboration module is used to control the information processing dimension D2 to remain unchanged and adjust the agent autonomy dimension D1 and the task allocation dimension D3 under the principle of minimum danger;

[0027] Among them, the principle of minimum danger is that for emergencies with a lower degree of danger, the autonomous dimension of the intelligent agent is adjusted first to keep the human-machine system at the original human-machine interaction level. For emergencies with a higher degree of danger, the task allocation between the pilot and the intelligent agent is adjusted by lowering the human-machine interaction level to improve the pilot's task participation.

[0028] Optionally, the adaptive collaboration module is used to control the task allocation dimension D3 to remain unchanged and adjust the agent autonomy dimension D1 and the information processing dimension D2 under the principle of giving priority to the agent's autonomy;

[0029] Among them, the principle of prioritizing the autonomous capabilities of the intelligent agent is to give priority to adjusting the autonomous dimensions of the intelligent agent to adapt to the current task allocation status and the requirements of information processing responsibilities. When adjusting the autonomous dimensions of the intelligent agent alone can no longer meet the needs of current task execution, the information processing responsibilities of the pilot and the intelligent agent are adjusted by changing the level of human-computer interaction.

[0030] Optionally, the acquisition terminal layer of the intelligent human-machine module group state monitoring module includes terminal devices for monitoring the physiological state of pilots, such as eye movement cameras, EEG amplifiers, ECG sensors, EMG sensors, and galvanic skin sensors, as well as terminal devices for monitoring the behavior state of pilots and intelligent agents, such as motion capture devices and operation logs;

[0031] The data acquisition layer of the intelligent human-machine module group status monitoring module processes the raw data collected by the terminal through the controller and sends out multi-modal physiological, motion data and instruction data with structured storage and timeline synchronization;

[0032] The feature extraction layer of the intelligent human-machine module group state monitoring module extracts corresponding physiological features, action features and instruction semantics;

[0033] The feature fusion layer of the intelligent human-machine module group state monitoring module realizes the fusion of physiological state features and behavioral state features, and combines the context of system state and task situation to realize further fusion of the two types of features;

[0034] The external interface layer of the intelligent human-machine module group state monitoring module provides external application interfaces for cognitive state assessment, interaction intention recognition, behavioral performance assessment, and human error recognition, providing a decision-making basis for the intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0035] A second aspect of the present invention provides an aircraft cockpit intelligent human-machine adaptive collaboration method, which is executed using a system as described in any one of the first aspects.

[0036] The present invention provides a functional architecture and method for an intelligent human-machine adaptive collaborative system for an aircraft cockpit, which has the advantages of hierarchical, modular, and reusable, can support the detailed design of the physical architecture of the intelligent human-machine system for the aircraft cockpit, and can provide a technical implementation path for efficient matching and dynamic natural interaction between human and machine in the aircraft cockpit. The intelligent human-machine adaptive collaborative method for an aircraft cockpit provided by the present invention can obtain the optimal intelligent human-machine adaptive collaborative solution for the aircraft cockpit by adjusting the human-machine interaction level, the autonomous dimension of the intelligent body, the information processing dimension, and the task allocation dimension based on the decision-making steps of the information flow diagram. The system functional architecture and method provided by the present invention can realize intelligent human-machine interaction that is adaptive to the physiological, psychological, behavioral state, and task requirements of the pilot, and can provide technical support for solving problems such as imbalanced cognitive load, loss of situational awareness, and frequent human errors of the pilot, enhancing the pilot's situation judgment and decision-making ability, and optimizing the comprehensive effectiveness of the cockpit human-machine system. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is the overall architecture diagram of the aircraft cockpit intelligent human-machine adaptive collaborative system;

[0038] Figure 2 This is the functional system architecture diagram of the intelligent human-machine module group status monitoring module;

[0039] Figure 3 To control the human-machine adaptive collaborative strategy to keep the autonomous dimension (D1) of the intelligent agent unchanged;

[0040] Figure 4 To control the human-machine adaptive collaborative strategy with constant information processing dimension (D2);

[0041] Figure 5 An adaptive human-machine collaborative strategy with constant dimension (D3) for control task allocation. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0043] The features and illustrative embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific settings and methods proposed below, but covers any improvements, replacements and modifications of structures, methods, devices without departing from the spirit of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity in the present invention.

[0044] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other, and the embodiments can refer to and quote each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0045] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0046] The present invention provides a functional architecture of an intelligent human-machine adaptive collaborative system for an aircraft cockpit, the functional architecture of the system comprising an intelligent human-machine module group, an intelligent body perception system module, an intelligent body decision system module, and an intelligent body execution system module;

[0047] The intelligent human-machine module group includes a display module, a state monitoring module, an adaptive collaboration module, and a control module. The functions of each module are connected in the logical order of perception, decision-making, and execution. The access data comes from the pilot's decision-making and execution behavior and the perception, decision-making, and execution system modules of the intelligent body. The output data affects the pilot's perception behavior.

[0048] The data sources of the above-mentioned state monitoring include physiological state data and behavioral state data. Physiological characteristics include physiological state parameters such as pilot eye movement, EEG, ECG, EMG, and GEP. Behavioral characteristics include behavioral state parameters of pilots and agents during human-computer interaction. The state monitoring module includes acquisition terminal layer, data acquisition layer, feature extraction layer, feature fusion layer, and external interface layer. Figure 2 As shown, where:

[0049] (1) The acquisition terminal layer includes terminal devices such as eye movement cameras, EEG amplifiers, ECG sensors, myoelectric sensors, and skin conductivity sensors that monitor the physiological status of pilots, as well as terminal devices such as motion capture devices and operation logs that monitor the behavioral status of pilots and intelligent agents;

[0050] (2) The data collection layer processes the raw data collected by the terminal through the controller and sends out multi-modal physiological, motion data and command data with structured storage and timeline synchronization;

[0051] (3) The feature extraction layer extracts the corresponding physiological features, action features and command semantics;

[0052] (4) The feature fusion layer realizes the fusion of physiological state features and behavioral state features, and combines the context of system state and task situation to achieve the fusion of the two types of features at a higher level;

[0053] (5) The external interface layer provides external application interfaces for cognitive state assessment, interaction intention recognition, behavioral performance assessment, and human error recognition, providing a decision-making basis for the intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0054] The above-mentioned adaptive collaboration module takes the state characteristics of the pilot and the intelligent agent as the decision-making basis, obtains the optimal solution for the intelligent human-machine adaptive collaboration in the aircraft cockpit based on the information flow diagram decision-making method, and realizes the adaptive adjustment of the macro human-machine interaction level and the micro variable degrees of freedom.

[0055] The intelligent agent perception system module accesses data from the flight mission environment, the aircraft body dynamics system, and the intelligent agent execution system module. The module obtains information such as the mission situation, the body state, and the intelligent agent execution results, and presents them to the pilot through the display module of the intelligent human-machine module group on the one hand, and provides basic input and decision-making basis for the intelligent agent decision system module on the other hand;

[0056] The intelligent agent decision system module accesses the data from the intelligent agent perception system module. After rule matching, knowledge reasoning and other steps, the module presents the decision result to the pilot through the display module of the intelligent human-machine module group on the one hand, and provides a basis for the deep processing of the state monitoring module, the adaptive collaboration module and the intelligent agent execution system module on the other hand;

[0057] The intelligent agent execution system module accesses data from the intelligent agent perception and decision-making system module. This module can realize human-machine team collaboration through the adaptive collaboration module of the intelligent human-machine module group, and can also directly control the aircraft body dynamics system through the control module of the intelligent human-machine module group. The execution result of this module will be returned to the intelligent agent perception system module.

[0058] The state monitoring module of the intelligent human-machine module group includes an acquisition terminal layer, a data acquisition layer, a feature extraction layer, a feature fusion layer, and an external interface layer. The data sources of the state monitoring include physiological state data and behavioral state data, wherein the physiological characteristics include physiological state parameters such as eye movement, electroencephalogram, electrocardiogram, electromyography, and skin electricity of the pilot, and the behavioral characteristics include behavioral state parameters of the pilot and the intelligent agent during the human-machine interaction process;

[0059] The present invention provides an aircraft cockpit intelligent human-machine adaptive collaboration method, comprising the following steps:

[0060] (1) Adopting the functional architecture of the aircraft cockpit intelligent human-machine adaptive collaborative system, which includes an intelligent human-machine module group, an intelligent agent perception system module, an intelligent agent decision system module, and an intelligent agent execution system module;

[0061] (2) Based on the characteristics of human-machine interaction tasks, discrete aircraft cockpit human-machine interaction levels are defined from a macro perspective to determine the overall boundaries of the human-machine collaboration mode;

[0062] The aircraft cockpit human-machine interaction levels include: L1, L2, L3, L4, and L5. The human-machine collaboration modes include: full human control, human-machine host and auxiliary, human-machine negotiation, machine-machine host and auxiliary, and full autonomy. The L1 interaction level corresponds to the "full human control" collaboration mode, the L2 interaction level corresponds to the "human-machine host and auxiliary" collaboration mode, the L3 interaction level corresponds to the "human-machine negotiation" collaboration mode, the L4 interaction level corresponds to the "machine-machine host and auxiliary" collaboration mode, and the L5 interaction level corresponds to the "full autonomy" collaboration mode.

[0063] (3) According to the level of autonomous capabilities of the intelligent agent, the internal attributes of the human-machine collaborative mode are determined from a microscopic perspective according to the control direction of different variable degrees of freedom;

[0064] The variable degrees of freedom include: agent autonomy dimension (D1), information processing dimension (D2), and task allocation dimension (D3);

[0065] (4) Using the pilot's cognitive load, situational awareness, human error and other physiological, psychological and behavioral states as trigger conditions, the aircraft cockpit human-machine interaction level is switched between L1 and L5 at the macro level, and the relative range of the intelligent agent autonomy dimension (D1), information processing dimension (D2) and task allocation dimension (D3) is adjusted at the micro level to establish an aircraft cockpit intelligent human-machine adaptive collaboration strategy;

[0066] Among them, the adaptive strategy of the agent's autonomy dimension is to adjust the boundary range of the agent's "ability to do" and "permission to do" from the perspective of capability and restriction respectively; the adaptive strategy of the information processing dimension is to adjust the responsibility allocation of the pilot and the agent in the vertical information processing stages such as information acquisition, information analysis, judgment and decision-making, and implementation of actions in accordance with the "observation-adjustment-decision-action (OODA)" loop; the adaptive strategy of the task allocation dimension is to adjust the task allocation and decision-making power attribution between the human-machine system and the cockpit according to the task environment, pilot workload, system performance and other states of the cockpit human-machine system;

[0067] (5) Keep the agent’s autonomy dimension (D1) unchanged, follow the principle of information processing priority, and adjust the information processing dimension (D3) and task allocation dimension (D2);

[0068] The information processing process priority principle is to prioritize the information processing processes of information acquisition, information analysis, judgment and decision-making, and action implementation of the task, and appropriately increase the information processing responsibilities of the pilot. When only adjusting the scope of pilot / agent responsibilities in the information processing dimension can no longer meet the needs of task execution, more or more difficult tasks are assigned to the pilot by reducing the level of human-computer interaction.

[0069] (6) Keep the information processing dimension (D2) unchanged, follow the principle of minimum danger, and adjust the agent autonomy dimension (D1) and task allocation dimension (D3);

[0070] The principle of minimum danger level is that, for emergencies with a lower degree of danger, the autonomous dimension of the agent is adjusted first to keep the human-machine system at the original human-machine interaction level. For emergencies with a higher degree of danger, the task allocation between the pilot and the agent is adjusted by reducing the human-machine interaction level to improve the pilot's task participation.

[0071] (7) The control task allocation dimension (D3) remains unchanged, and the agent autonomy dimension (D1) and information processing dimension (D2) are adjusted in accordance with the principle of giving priority to the agent's autonomy.

[0072] The principle of priority of the autonomous capability of the intelligent agent is to give priority to adjusting the autonomous dimension of the intelligent agent to adapt to the current task allocation status and the requirements of information processing responsibilities. When the requirements of the current task execution cannot be met by adjusting the autonomous dimension of the intelligent agent alone, the information processing responsibilities of the pilot and the intelligent agent are adjusted by changing the human-machine interaction level.

[0073] (8) For all feasible macro-level and micro-dimensional combination adjustment schemes, the optimal aircraft cockpit intelligent human-machine adaptive collaborative solution is obtained based on the information flow diagram decision-making method.

[0074] Furthermore, the specific steps of obtaining the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit based on the information flow diagram decision method are as follows:

[0075] 1) Determine the feasible adjustment solution set S = {s 1 ,s 2 ,…,s n}, where n represents the total number of feasible solutions, and each element in the set S represents a feasible solution;

[0076] 2) Determine the comprehensive effectiveness evaluation index set of the human-machine system F = {f1 ,f 2 ,…,f m}, where m represents the total number of influencing factors, and each element in the set F represents an influencing factor;

[0077] 3) Under the measurement of the evaluation index set F, compare all feasible solutions in the set S two by two and determine the score set R = {r 1 ,r 2 ,…,r m}, weight set Ω = {ω 1 ,ω 2 ,…,ω m}, each value in the score set and weight set corresponds to the element of the evaluation index set, and the "+" and "-" symbols are used to indicate positive correlation and negative correlation respectively. The results of pairwise comparison are l ij The linear weighting method yields:

[0078] l ij =ω 1 r 1 +ω 2 r 2 +Lω m r m

[0079] Among them, i and j represent two feasible solutions in the set S (1≤i≤n,1≤j≤n), l ij If it is a positive number, it means that solution i is better than solution j. ij If it is a negative number, it means that plan i is worse than plan j. ij If it is 0, it means that scheme i and scheme j are equally good;

[0080] 4) Treat each feasible solution as an independent node, and connect each solution with an edge to form a complete information flow graph. ij The absolute value of represents the information flow load of each edge, expressed as l ij The "+" and "-" signs determine the direction of the edge (the arrow points to the "+" direction), l ij If it is 0, the edge can point to any direction;

[0081] 5) Calculate the information flow load of node i in the complete information flow graph

[0082] 6) Sort all nodes in the complete information flow graph according to the size of the information flow load. The node with a larger information flow load has a higher priority for the corresponding feasible solution, thereby determining the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit.

[0083] The above description is only a further embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and concepts of the present invention within the scope disclosed by the present invention, which belong to the protection scope of the present invention.

Claims

1. An aircraft cockpit intelligent human-machine adaptive collaborative system, characterized in that: include: Intelligent human-machine module group, intelligent agent perception system module, intelligent agent decision system module, and intelligent agent execution system module; The intelligent human-machine module group includes: a display module, a state monitoring module, an adaptive collaboration module, and a control module. The functions of each module are connected in the logical order of perception, decision-making, and execution. The access data comes from the pilot's decision-making and execution behavior and the perception, decision-making, and execution system modules of the intelligent body. The output data affects the pilot's perception behavior. The intelligent agent perception system module accesses data from the flight mission environment, the aircraft body dynamics system, and the intelligent agent execution system module. The module obtains the mission situation, body status, and intelligent agent execution results, and presents them to the pilot through the display module of the intelligent human-machine module group on the one hand, and provides basic input and decision-making basis for the intelligent agent decision system module on the other hand; The intelligent agent decision system module accesses the data from the intelligent agent perception system module. After rule matching and knowledge reasoning, the module presents the decision results to the pilot through the display module of the intelligent human-machine module group on the one hand, and provides a basis for the deep processing of the state monitoring module, the adaptive collaboration module, and the intelligent agent execution system module on the other hand; The intelligent agent execution system module accesses data from the intelligent agent perception and decision-making system module, and is used to realize human-machine team collaboration through the adaptive collaboration module of the intelligent human-machine module group, and / or directly control the aircraft body dynamics system through the control module of the intelligent human-machine module group. The execution result of this module will be returned to the intelligent agent perception system module; The state monitoring module of the intelligent human-machine module group includes an acquisition terminal layer, a data acquisition layer, a feature extraction layer, a feature fusion layer, and an external interface layer. The data sources of the state monitoring include physiological state data and behavioral state data, wherein the physiological characteristics include physiological state parameters such as eye movement, electroencephalogram, electrocardiogram, electromyography, and skin electricity of the pilot, and the behavioral characteristics include behavioral state parameters of the pilot and the intelligent agent during the human-machine interaction process; The adaptive collaboration module of the intelligent human-machine module group takes the state characteristics of the pilot and the intelligent body as the decision-making basis, obtains the optimal solution for the intelligent human-machine adaptive collaboration in the aircraft cockpit based on the information flow diagram decision-making method, and realizes the adaptive adjustment of the macro human-machine interaction level and the micro variable degree of freedom.

2. The aircraft cockpit intelligent human-machine adaptive collaborative system according to claim 1, characterized in that: The adaptive collaboration module is used to switch the cockpit human-machine interaction level between the full human-controlled interaction level L1, the human-host-assistant interaction level L2, the human-machine negotiation interaction level L3, the aircraft-host-assistant interaction level L4, and the fully autonomous interaction level L5 based on the pilot's physiological, psychological, and behavioral states as trigger conditions. At the micro level, the relative ranges of the intelligent agent autonomy dimension D1, the information processing dimension D2, and the task allocation dimension D3 are adjusted according to at least one of the information processing process priority principle, the intelligent agent autonomy capability priority principle, and the minimum danger level principle, and the aircraft cockpit intelligent human-machine adaptive collaboration strategy is established; wherein, The pilot's physiological, psychological, and behavioral states include pilot cognitive load, situational awareness, and human errors.

3. The aircraft cockpit intelligent human-machine adaptive collaborative system according to claim 2, characterized in that: The adaptive collaboration module is used to perform the following steps: 1) Determine a feasible adaptive collaborative adjustment solution set S = {s1, s2, ..., s n }, where n represents the total number of feasible solutions, and each element in the set S represents a feasible solution; 2) Determine the performance evaluation index set F = {f1,f2,…,f m }, where m represents the total number of influencing factors, and each element in the set F represents an influencing factor; 3) Under the measurement of the performance evaluation index set F, compare all feasible solutions in the set S two by two and determine the score set R = {r1, r2, …, r m }, weight set Ω={ω1,ω2,…,ω m }, each value in the score set and weight set corresponds to the element of the evaluation index set, and the "+" and "-" symbols are used to indicate positive correlation and negative correlation respectively. The results of pairwise comparison are l ij The linear weighting method yields: l ij =ω1r1+ω2r2+Lω m r m Among them, i and j represent two feasible solutions in the set S, 1≤i≤n, 1≤j≤n, l ij If it is a positive number, it means that solution i is better than solution j. ij If it is a negative number, it means that plan i is worse than plan j. ij If it is 0, it means that scheme i and scheme j are equally good; 4) Treat each feasible solution as an independent node, and connect each solution with an edge to form a complete information flow graph. ij The absolute value of represents the information flow load of each edge, expressed as l ij The "+" and "-" signs determine the direction of the edge, and the arrow points to the "+" direction. ij If it is 0, the edge can point to any direction; 5) Calculate the information flow load of node i in the complete information flow graph 6) Sort all nodes in the complete information flow graph according to the size of the information flow load. The node with a larger information flow load has a higher priority for the corresponding feasible solution, thereby determining the optimal solution for intelligent human-machine adaptive collaboration in the aircraft cockpit.

4. The aircraft cockpit intelligent human-machine adaptive collaborative system according to claim 2, characterized in that: The adaptive collaboration module is used to control the agent's autonomy dimension D1 unchanged and adjust the information processing dimension D3 and the task allocation dimension D2 under the principle of information processing priority; Among them, the information processing process priority principle is to prioritize the information acquisition, information analysis, judgment and decision-making, and action implementation of the task, appropriately increase the pilot's information processing responsibilities, and when only adjusting the scope of pilot / agent responsibilities in the information processing dimension can no longer meet the needs of task execution, by reducing the level of human-computer interaction, more or more difficult tasks will be assigned to the pilot for processing.

5. The aircraft cockpit intelligent human-machine adaptive collaborative system according to claim 2, characterized in that: The adaptive collaboration module is used to control the information processing dimension D2 to remain unchanged and adjust the agent autonomy dimension D1 and task allocation dimension D3 under the principle of minimum danger; Among them, the principle of minimum danger is that for emergencies with a lower degree of danger, the autonomous dimension of the intelligent agent is adjusted first to keep the human-machine system at the original human-machine interaction level. For emergencies with a higher degree of danger, the task allocation between the pilot and the intelligent agent is adjusted by lowering the human-machine interaction level to improve the pilot's task participation.

6. The aircraft cockpit intelligent human-machine adaptive collaborative system according to claim 2, characterized in that: The adaptive collaboration module is used to control the task allocation dimension D3 unchanged and adjust the agent autonomy dimension D1 and information processing dimension D2 under the principle of agent autonomy priority; Among them, the principle of prioritizing the autonomous capabilities of the intelligent agent is to give priority to adjusting the autonomous dimensions of the intelligent agent to adapt to the current task allocation status and the requirements of information processing responsibilities. When adjusting the autonomous dimensions of the intelligent agent alone can no longer meet the needs of current task execution, the information processing responsibilities of the pilot and the intelligent agent are adjusted by changing the level of human-computer interaction.

7. The aircraft cockpit intelligent human-machine adaptive collaborative system according to claim 1, characterized in that: The acquisition terminal layer of the intelligent human-machine module group state monitoring module includes terminal devices for monitoring the physiological state of pilots, such as eye movement cameras, EEG amplifiers, ECG sensors, EMG sensors, and galvanic skin sensors, as well as terminal devices for monitoring the behavior state of pilots and intelligent agents, such as motion capture devices and operation logs; The data acquisition layer of the intelligent human-machine module group status monitoring module processes the raw data collected by the terminal through the controller and sends out multi-modal physiological, motion data and instruction data with structured storage and timeline synchronization; The feature extraction layer of the intelligent human-machine module group state monitoring module extracts corresponding physiological features, action features and instruction semantics; The feature fusion layer of the intelligent human-machine module group state monitoring module realizes the fusion of physiological state features and behavioral state features, and combines the context of system state and task situation to realize further fusion of the two types of features; The external interface layer of the intelligent human-machine module group state monitoring module provides external application interfaces for cognitive state assessment, interaction intention recognition, behavioral performance assessment, and human error recognition, providing a decision-making basis for the intelligent human-machine adaptive collaboration in the aircraft cockpit.

8. An aircraft cockpit intelligent human-machine adaptive collaboration method, characterized in that: The method is implemented by using a system as claimed in any one of claims 1 to 7.

Images