Airplane cockpit multi-channel man-machine intelligent interaction system and method

Abstract

The invention provides an airplane cockpit multi-channel man-machine intelligent interaction system and method. The system comprises a multi-channel data access module, a multi-channel interaction instruction matching module, a multi-channel relay communication function module, a multi-channel priority ranking module, a multi-channel interaction information adjusting module and a cockpit display control interface module. Natural cooperation and efficiency optimization of multi-channel interaction modes such as touch control, voice, eye movement, gestures and electroencephalogram in an aircraft cockpit interaction environment are achieved, a scientific scheme is provided for organizing aircraft cockpit multi-channel man-machine intelligent interaction information, and an important basis is provided for establishing an aircraft cockpit multi-channel man-machine intelligent interaction system architecture. And the design and development of the multi-channel man-machine intelligent interaction system of the aircraft cockpit can be effectively supported.

Description

Technical Field

[0001] The present invention belongs to the technical field of complex system human-computer interaction, and relates to a multi-channel human-computer intelligent interaction system and method for an aircraft cockpit. Background Art

[0002] Multi-channel human-computer intelligent interaction is to turn the human-computer interaction process into knowledge, including knowledge acquisition, knowledge representation, knowledge reasoning and other links, from which human thinking patterns are mined and extracted, and represented in a specific form, making them objects that can be operated by computers, so that complex systems have certain human intelligence. The development of human-computer intelligent interaction systems is reflected in information input / output devices, operation feedback methods, interface visualization methods, human-computer collaboration, etc. With the further development of artificial intelligence technology, touch interaction, voice interaction, eye movement interaction, somatosensory interaction, mixed reality interaction, brain-computer interaction, human-intelligent interaction and other technologies are becoming the most novel design features of human-computer interaction systems.

[0003] At present, in the aviation field, the research on touch interaction technology and voice interaction technology is relatively mature and has been successfully applied to the cockpits of some series of aircraft. The cockpit human-machine system integrates flight display, flight planning, airborne system control and other control functions. By using direct voice input and touch control, the workload of pilots is effectively reduced. Multi-channel human-machine intelligent interaction will become the most important human-machine interaction mode in the cockpit of future aircraft. The aircraft cockpit that integrates multi-channel human-machine intelligent interaction technology has the advantages of diversified interaction channels, dynamic function allocation, and coordinated human-machine decision-making. However, there are also new problems that have not been solved. The main problem is that the system architecture of the multi-channel human-machine intelligent interaction system for the aircraft cockpit has not been established, and there is no scientific solution for how to organize the multi-channel human-machine intelligent interaction information. There is a lack of effective methods to support the design of the multi-channel human-machine intelligent interaction system for the aircraft cockpit. Summary of the invention

[0004] In view of the current lack of technical means to effectively support the design of a multi-channel human-machine intelligent interaction system for an aircraft cockpit, the present invention provides a multi-channel human-machine intelligent interaction system and method for an aircraft cockpit. The system and method can reasonably adjust the information flow load of key nodes of the multi-channel human-machine intelligent interaction, thereby providing effective technical support for the design and optimization of the multi-channel human-machine intelligent interaction system for an aircraft cockpit.

[0005] A first aspect of the present invention provides an aircraft cockpit multi-channel human-machine intelligent interaction system, comprising: a multi-channel data access module, a multi-channel interaction instruction matching module, a multi-channel relay communication function module, a multi-channel priority sorting module, a multi-channel interaction information adjustment module, and a cockpit display control interface module;

[0006] The multi-channel data access module takes the pilot's physiological and behavioral data as input, extracts basic feature values ​​of the physiological and behavioral data through time-frequency analysis and pattern matching, and the basic feature values ​​include touch coordinates, intonation and speech speed, gaze duration, body posture, and brain area activation, and further extracts fusion feature values ​​from the basic feature values, and the fusion feature values ​​include target icons, hot word semantics, target area, gesture semantics, and cognitive load, and transmits the fusion feature values ​​to the multi-channel interactive instruction matching module;

[0007] The multi-channel interactive instruction matching module matches the control semantics of the fused feature value according to the mapping relationship between the fused feature value and the control semantics, and transmits all successfully matched control semantics to the multi-channel relay communication function module; the control semantics include: switch type continuous adjustment of the controlled object state type control, flight control type continuous adjustment of the controlled object state type control, discrete adjustment of the controlled object state type control

[0008] The multi-channel relay communication function module classifies the received control semantics according to the trigger condition combination, stores the control semantics that arrive at the module first according to the order in which the control semantics arrive at the module, combined with the current mission phase or system status of the aircraft, and generates a control instruction to be transmitted to the multi-channel priority sorting module when all the trigger conditions for triggering the control operation are met;

[0009] The multi-channel priority sorting module evaluates and determines the importance and conflict possibility of the multi-channel control instructions in combination with the current mission phase or system status of the aircraft. For control instructions without conflicts, the instructions are sorted by importance to generate a feasible control sequence. For control instructions with conflicts, only the instructions with the highest importance are retained to sort and generate a feasible control sequence, and the generated feasible control sequence is transmitted to the multi-channel interactive information adjustment module.

[0010] The multi-channel interactive information adjustment module analyzes different feasible control sequences, takes human-machine interactive objects as nodes, generates directed edges according to the controlling or controlled relationship, takes the relationship complexity between different nodes as the weight of the directed edge, constructs a multi-channel human-machine intelligent interactive information flow network for the aircraft cockpit, compares the information flow load of the pilot node in different information flow networks, selects the feasible control sequence with the lowest information flow intensity of the pilot node as the optimal control solution, and transmits it to the cockpit display control interface module;

[0011] The cockpit display control interface module provides all display control interfaces for flight control function, flight management function, system monitoring function, system control function and communication function in the aircraft cockpit. According to the optimal control scheme generated by the multi-channel interactive information adjustment module, the control signal is transmitted to each system interface in the optimal sequence for execution.

[0012] Optionally, the switch-type continuous adjustment of the controlled object state type control of the multi-channel interactive instruction matching module refers to continuous knob operation using a combination of tactile serial operation and tactile activation, including an interactive scene of setting a permitted altitude on the flight control panel;

[0013] The flight control type of the multi-channel interactive command matching module continuously adjusts the state of the controlled object, which refers to continuous handle operation in a tactile activation manner, including an interactive scenario of disconnecting the automatic thrust during the automatic landing phase and pushing the thrust handle to the slow position;

[0014] The discrete adjustment controlled object state type control of the multi-channel interactive instruction matching module refers to the use of tactile activated relaxation to perform button operations, discrete knob operations, and discrete handle operations, including interactive scenarios such as switching navigation modes on the control panel, turning on the landing light switch, and using the speed brake handle to adjust the descent rate with different altitudes / speeds.

[0015] Optionally, the triggering conditions for the multi-channel relay communication function module to trigger the control operation include flight parameters and status information, and provide a logical operator connection, and generate a control instruction when the trigger condition judgment statement is true; the flight parameters include: flight altitude, airspeed, heading, attitude angle; the status information includes: landing gear position, flap position, switch position;

[0016] Logical operations include: logical AND, logical OR, and logical NOT.

[0017] Optionally, the multi-channel priority sorting module ranks the importance of multi-channel control instructions based on the system-level safety assessment results during the aircraft development phase, including functional hazard assessment, preliminary system safety assessment, and common cause analysis, based on the potential possibility of unexpected functional failure / malfunction within the aircraft system, or unexpected functional failure / malfunction between related systems, combined with the severity of different consequences such as catastrophic, dangerous, large impact, small impact, and no safety impact.

[0018] Optionally, the adjustment strategy for the information flow intensity of the pilot node of the multi-channel interactive information adjustment module is to adopt tactile serial operation, tactile activation, and non-contact interaction methods such as voice, eye movement, and gesture from the perspective of information simplification and intelligence.

[0019] Optionally, the pilot's physiological and behavioral data include: touch, voice, eye movement, gesture, and EEG.

[0020] Optionally, the functional system architecture of the multi-channel data access module includes a collection terminal, a data layer, a feature layer, a fusion layer and an external interface;

[0021] At the acquisition terminal layer, the terminals for monitoring the pilot's physiological state include eye movement cameras, EEG amplifiers, ECG sensors, and EMG sensors; the terminals for monitoring the pilot's behavioral state include voice recorders, gesture capturers, and touch recorders;

[0022] At the data collection layer, the raw data collected by the terminal is processed by the controller and sent out as structured storage, timeline-synchronized multi-source physiological data, multi-modal human-computer interaction action data, and instruction data;

[0023] In the feature extraction layer, the corresponding physiological state features, behavioral state features and corresponding instruction semantics are extracted;

[0024] In the feature fusion layer, physiological state feature fusion and behavioral state feature fusion are realized, and the situational context of system state and task state is combined to realize further fusion of the two types of features;

[0025] At the external interface layer, combined with task requirements, the functional architecture of this module provides output interfaces for target icons, hot word semantics, target areas, gesture semantics, and cognitive load, while also connecting to a multi-channel interactive command matching module.

[0026] A second aspect of the present invention provides a multi-channel human-machine intelligent interaction method for an aircraft cockpit, which is executed using a system as described in any one of the first aspects.

[0027] The present invention provides a multi-channel human-machine intelligent interaction system and method for an aircraft cockpit, which realizes the natural coordination and efficiency optimization of multi-channel interaction modes such as touch, voice, eye movement, gesture, and electroencephalogram in the interactive environment of the aircraft cockpit, provides a scientific solution for organizing multi-channel human-machine intelligent interaction information in the aircraft cockpit, provides an important basis for establishing the system architecture of the multi-channel human-machine intelligent interaction system for the aircraft cockpit, and can effectively support the design and development of the multi-channel human-machine intelligent interaction system for the aircraft cockpit. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is the structural diagram of the intelligent human-machine system in the aircraft cockpit;

[0029] Figure 2 It is a multi-channel human-machine intelligent interaction circuit diagram for the aircraft cockpit;

[0030] Figure 3 It is the functional system architecture of the multi-channel data access module. DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0032] The features and illustrative embodiments of various aspects of the present invention will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention. The present invention is by no means limited to any specific settings and methods proposed below, but covers any improvements, replacements and modifications of structures, methods, devices without departing from the spirit of the present invention. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid unnecessary ambiguity in the present invention.

[0033] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other, and the embodiments can refer to and quote each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0034] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0035] The present invention provides a multi-channel human-machine intelligent interaction method for an aircraft cockpit, comprising the following steps:

[0036] (1) A multi-channel human-machine intelligent interaction system for an aircraft cockpit is used, which includes a multi-channel data access module, a multi-channel interaction instruction matching module, a multi-channel relay communication function module, a multi-channel priority sorting module, a multi-channel interaction information adjustment module, and a cockpit display control interface module;

[0037] (2) Collect the pilot's physiological and behavioral data such as touch, voice, eye movement, gesture, and EEG generated during the human-machine interaction process through sensors and terminal devices, transfer the raw data to the multi-channel data access module, extract the basic feature values ​​of different data and fuse them, and transfer the fused feature values ​​to the multi-channel interaction command matching module;

[0038] (3) Establishing a mapping relationship from fusion feature values ​​to control semantics through a multi-channel interactive instruction matching module, and transmitting all successfully matched control semantics to a multi-channel relay communication function module;

[0039] (4) The received control semantics are classified by conditional combination through the multi-channel relay communication function module, and when all conditions for triggering the control operation are met, a control instruction is generated and transmitted to the multi-channel priority sorting module;

[0040] (5) evaluating and judging the importance and conflict possibility of multi-channel control instructions through a multi-channel priority sorting module, generating a feasible control sequence by sorting the importance, and transmitting the generated feasible control sequence to a multi-channel interactive information adjustment module;

[0041] (6) Analyze different feasible control sequences through the multi-channel interactive information adjustment module, construct a multi-channel human-machine intelligent interactive information flow network for the aircraft cockpit, compare the information flow load of the pilot node in different information flow networks based on the complexity of the relationship between different nodes, select the feasible control sequence with the lowest information flow intensity of the pilot node as the optimal control solution, and transmit the optimal control solution to the cockpit display control interface module;

[0042] (7) The control signal is transmitted to each airborne system interface through the cockpit display control interface module to perform flight control, flight management, system monitoring, system control, communication and other functions, thereby realizing multi-channel human-machine intelligent interactive closed-loop control in the aircraft cockpit.

[0043] like Figure 1 As shown in the figure, in the structure of the intelligent human-machine system in the aircraft cockpit, multiple interactive channels such as touch, voice, eye movement, gesture, and EEG are located at the interface between the pilot and the airborne system, and play an intermediary role between the pilot's receptors, effectors and the airborne system. The aircraft cockpit multi-channel human-machine intelligent interaction system is a general term for the cockpit sending device, receiving device and multi-channel human-machine interface.

[0044] like Figure 2 As shown, the aircraft cockpit multi-channel human-machine intelligent interaction loop involves pilots, airborne systems, and a multi-channel human-machine intelligent interaction system, wherein the aircraft cockpit multi-channel human-machine intelligent interaction system includes a multi-channel data access module, a multi-channel interaction instruction matching module, a multi-channel relay communication function module, a multi-channel priority sorting module, a multi-channel interaction information adjustment module, and a cockpit display control interface module.

[0045] The multi-channel data access module takes the pilot's physiological and behavioral data such as touch, voice, eye movement, gesture, and EEG as input, and extracts the basic feature values ​​of different data through time-frequency analysis, pattern matching and other means, including touch coordinates, voice intonation and speed, gaze duration, body posture, brain area activation, etc., and further extracts high-level fusion feature values ​​from the basic feature values, including target icons, hot word semantics, target area, gesture semantics, cognitive load, etc., and transmits the fusion feature values ​​to the multi-channel interactive command matching module.

[0046] Among them, the functional system architecture of the multi-channel data access module is as follows: Figure 3 As shown, it includes acquisition terminal, data layer, feature layer, fusion layer and external interface;

[0047] At the acquisition terminal layer, the terminals for monitoring the pilot's physiological state include eye movement cameras, EEG amplifiers, ECG sensors, and EMG sensors. The terminals for monitoring the pilot's behavioral state include voice recorders, gesture capturers, touch recorders, and other behavior capture / recording devices.

[0048] At the data collection layer, the raw data collected by the terminal is processed by the controller and sent out as structured storage, timeline-synchronized multi-source physiological data, multi-modal human-computer interaction action data, and instruction data;

[0049] In the feature extraction layer, the corresponding physiological state features, behavioral state features and corresponding instruction semantics are extracted;

[0050] At the feature fusion layer, physiological state feature fusion and behavioral state feature fusion are realized, and the situational context of system state and task state are combined to realize the fusion of the two types of features at a higher level;

[0051] At the external interface layer, combined with task requirements, the functional architecture of this module provides output interfaces for target icons, hot word semantics, target areas, gesture semantics, cognitive load and other features, while also connecting to a multi-channel interactive command matching module.

[0052] The multi-channel interactive instruction matching module establishes a mapping relationship from fusion characteristic values ​​to control semantics, including continuous adjustment (switch type) controlled object state type control, continuous adjustment (flight control type) controlled object state type control, and discrete adjustment controlled object state type control, and transmits all successfully matched control semantics to the multi-channel relay communication function module.

[0053] Among them, the continuous adjustment (switch type) controlled object state control includes a single control mode of tactile continuous adjustment, a combined control mode of tactile serial operation and tactile activation, and a continuous knob operation mode, which is suitable for interactive scenarios such as setting the permitted altitude on the flight control panel.

[0054] Among them, the continuous adjustment (flight control type) controlled object state type control includes tactile continuous adjustment control method, tactile activation control method and continuous handle operation method, which is suitable for interactive scenarios such as disconnecting automatic thrust during automatic landing phase and pushing the thrust handle to the slow position.

[0055] Among them, the discrete adjustment of the controlled object state type control includes tactile activation control method, tactile discrete adjustment control method, as well as dial operation, discrete knob operation, discrete handle operation and other methods, which are suitable for interactive scenarios such as switching the navigation mode on the electronic flight instrument system control panel during the descent preparation phase, turning on the landing light switch during the descent phase, and using the speed brake handle to adjust the descent rate with different altitudes / speeds.

[0056] The multi-channel relay communication function module classifies the received control semantics according to conditional combinations, and stores the control semantics that arrive at the module first according to the order in which the control semantics arrive at the module and the current mission phase or system status of the aircraft. When all conditions for triggering the control operation are met, a control instruction is generated and transmitted to the multi-channel priority sorting module.

[0057] The multi-channel priority sorting module evaluates and determines the importance and conflict possibility of multi-channel control instructions in combination with the current mission stage or system status of the aircraft. For control instructions that do not conflict, a feasible control sequence is generated by sorting them by importance. For control instructions that conflict, only the instructions with the highest importance are retained to sort and generate a feasible control sequence, and the generated feasible control sequence is passed to the multi-channel interactive information adjustment module.

[0058] The multi-channel interactive information adjustment module analyzes different feasible control sequences, takes human-machine interaction objects as nodes, generates directed edges according to the controlling or controlled relationship, takes the relationship complexity between different nodes as the weight of the directed edge, constructs a multi-channel human-machine intelligent interactive information flow network for the aircraft cockpit, compares the information flow load of the pilot node in different information flow networks, selects the feasible control sequence with the lowest information flow intensity of the pilot node as the optimal control solution, and transmits it to the cockpit display control interface module.

[0059] The cockpit display control interface module provides all display control interfaces in the aircraft cockpit, including flight control function, flight management function, system monitoring function, system control function, communication function, etc. According to the optimal control scheme generated by the multi-channel interactive information adjustment module, the control signal is transmitted to each system interface in the optimal sequence for execution.

[0060] A multi-channel human-machine intelligent interaction method for an aircraft cockpit is as follows:

[0061] 1. Collect the pilot's physiological and behavioral data such as touch, voice, eye movement, gesture, and EEG generated during the human-machine interaction process through sensors and terminal devices, transfer the original data to the multi-channel data access module, extract the basic feature values ​​of different data and fuse them, and transfer the fused feature values ​​to the multi-channel interaction command matching module;

[0062] 2. Establish a mapping relationship from fusion feature values ​​to control semantics through the multi-channel interactive instruction matching module, and transfer all successfully matched control semantics to the multi-channel relay communication function module;

[0063] 3. The received control semantics are classified by conditional combination through the multi-channel relay communication function module. When all the conditions for triggering the control operation are met, the control instructions are generated and passed to the multi-channel priority sorting module;

[0064] 4. The importance and conflict possibility of multi-channel control instructions are evaluated and judged through the multi-channel priority sorting module, and a feasible control sequence is generated by sorting the importance, and the generated feasible control sequence is passed to the multi-channel interactive information adjustment module;

[0065] 5. Analyze different feasible control sequences through the multi-channel interactive information adjustment module, build a multi-channel human-machine intelligent interactive information flow network for the aircraft cockpit, compare the information flow load of the pilot node in different information flow networks based on the complexity of the relationship between different nodes, select the feasible control sequence with the lowest information flow intensity of the pilot node as the optimal control solution, and transmit the optimal control solution to the cockpit display control interface module;

[0066] 6. The control signal is transmitted to each airborne system interface through the cockpit display control interface module to perform flight control, flight management, system monitoring, system control, communication and other functions, realizing multi-channel human-machine intelligent interactive closed-loop control of the aircraft cockpit.

[0067] The above description is only a further embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solutions and concepts of the present invention within the scope disclosed by the present invention, which belong to the protection scope of the present invention.

Claims

1. An aircraft cockpit multi-channel human-machine intelligent interaction system, characterized in that: include: Multi-channel data access module, multi-channel interactive command matching module, multi-channel relay communication function module, multi-channel priority sorting module, multi-channel interactive information adjustment module, cockpit display control interface module; The multi-channel data access module takes the pilot's physiological and behavioral data as input, extracts basic feature values ​​of the physiological and behavioral data through time-frequency analysis and pattern matching, and the basic feature values ​​include touch coordinates, intonation and speech speed, gaze duration, body posture, and brain area activation, and further extracts fusion feature values ​​from the basic feature values, and the fusion feature values ​​include target icons, hot word semantics, target area, gesture semantics, and cognitive load, and transmits the fusion feature values ​​to the multi-channel interactive instruction matching module; The multi-channel interactive instruction matching module matches the control semantics of the fused feature value according to the mapping relationship between the fused feature value and the control semantics, and transmits all successfully matched control semantics to the multi-channel relay communication function module; the control semantics include: switch type continuous adjustment of the controlled object state type control, flight control type continuous adjustment of the controlled object state type control, discrete adjustment of the controlled object state type control The multi-channel relay communication function module classifies the received control semantics according to the trigger condition combination, stores the control semantics that arrive at the module first according to the order in which the control semantics arrive at the module, combined with the current mission phase or system status of the aircraft, and generates a control instruction to be transmitted to the multi-channel priority sorting module when all the trigger conditions for triggering the control operation are met; The multi-channel priority sorting module evaluates and determines the importance and conflict possibility of the multi-channel control instructions in combination with the current mission phase or system status of the aircraft. For control instructions without conflicts, the instructions are sorted by importance to generate a feasible control sequence. For control instructions with conflicts, only the instructions with the highest importance are retained to sort and generate a feasible control sequence, and the generated feasible control sequence is transmitted to the multi-channel interactive information adjustment module. The multi-channel interactive information adjustment module analyzes different feasible control sequences, takes human-machine interactive objects as nodes, generates directed edges according to the controlling or controlled relationship, takes the relationship complexity between different nodes as the weight of the directed edge, constructs a multi-channel human-machine intelligent interactive information flow network for the aircraft cockpit, compares the information flow load of the pilot node in different information flow networks, selects the feasible control sequence with the lowest information flow intensity of the pilot node as the optimal control solution, and transmits it to the cockpit display control interface module; The cockpit display control interface module provides all display control interfaces for flight control function, flight management function, system monitoring function, system control function and communication function in the aircraft cockpit. According to the optimal control scheme generated by the multi-channel interactive information adjustment module, the control signal is transmitted to each system interface in the optimal sequence for execution.

2. The aircraft cockpit multi-channel human-machine intelligent interaction system according to claim 1, characterized in that: The switch-type continuous adjustment of the controlled object state type control of the multi-channel interactive command matching module refers to continuous knob operation using a combination of tactile serial operation and tactile activation, including an interactive scene for setting a permitted altitude on the flight control panel; The flight control type of the multi-channel interactive command matching module continuously adjusts the state of the controlled object, which refers to continuous handle operation in a tactile activation manner, including an interactive scenario of disconnecting the automatic thrust during the automatic landing phase and pushing the thrust handle to the slow position; The discrete adjustment controlled object state type control of the multi-channel interactive instruction matching module refers to the use of tactile activated relaxation to perform button operations, discrete knob operations, and discrete handle operations, including interactive scenarios such as switching navigation modes on the control panel, turning on the landing light switch, and using the speed brake handle to adjust the descent rate with different altitudes / speeds.

3. The aircraft cockpit multi-channel human-machine intelligent interaction system according to claim 1, characterized in that: The triggering conditions for triggering the control operation of the multi-channel relay communication function module include flight parameters and status information, and provide logical operator connection, and generate control instructions when the triggering condition judgment statement is true; Flight parameters include: flight altitude, airspeed, heading, attitude angle; status information includes: landing gear position, flap position, switch position; Logical operations include: logical AND, logical OR, and logical NOT.

4. The aircraft cockpit multi-channel human-machine intelligent interaction system according to claim 1, characterized in that: The multi-channel priority sorting module ranks the importance of multi-channel control instructions based on the system-level safety assessment results in the aircraft development stage, including functional hazard assessment, preliminary system safety assessment, and common cause analysis, based on the potential possibility of unexpected functional failure / malfunction within the aircraft system, or unexpected functional failure / malfunction between related systems, combined with the severity of different consequences such as catastrophic, dangerous, large impact, small impact, and no safety impact.

5. The aircraft cockpit multi-channel human-machine intelligent interaction system according to claim 1, characterized in that: The adjustment strategy of the pilot node information flow intensity of the multi-channel interactive information adjustment module is to adopt tactile serial operation, tactile activation and non-contact interaction methods such as voice, eye movement and gesture from the perspective of information simplification and intelligence.

6. The aircraft cockpit multi-channel human-machine intelligent interaction system according to claim 1, characterized in that: Pilots’ physiological and behavioral data include touch, voice, eye movement, gestures, and EEG.

7. The aircraft cockpit multi-channel human-machine intelligent interaction system according to claim 6, characterized in that: The functional system architecture of the multi-channel data access module includes a collection terminal, a data layer, a feature layer, a fusion layer and an external interface; At the acquisition terminal layer, the terminals for monitoring the pilot's physiological state include eye movement cameras, EEG amplifiers, ECG sensors, and EMG sensors; the terminals for monitoring the pilot's behavioral state include voice recorders, gesture capturers, and touch recorders; At the data collection layer, the raw data collected by the terminal is processed by the controller and sent out as structured storage, timeline-synchronized multi-source physiological data, multi-modal human-computer interaction action data, and instruction data; In the feature extraction layer, the corresponding physiological state features, behavioral state features and corresponding instruction semantics are extracted; In the feature fusion layer, physiological state feature fusion and behavioral state feature fusion are realized, and the situational context of system state and task state is combined to realize further fusion of the two types of features; At the external interface layer, combined with task requirements, the functional architecture of this module provides output interfaces for target icons, hot word semantics, target areas, gesture semantics, and cognitive load, while also connecting to a multi-channel interactive command matching module.

8. A multi-channel human-machine intelligent interaction method for an aircraft cockpit, characterized in that: The method is implemented by using a system as claimed in any one of claims 1 to 7.

Images