Man-machine cooperative flight multi-dimensional cognitive state quantitative evaluation method for low-altitude aircraft

Abstract

The invention discloses a man-machine cooperative flight multi-dimensional cognitive state quantitative evaluation method for a low-altitude aircraft, and the method specifically comprises the steps: synchronously collecting the electroencephalogram, eye movement, skin electricity, heart rate and other multi-dimensional physiological data of a driver and the flight data of the aircraft; based on coupling of electroencephalogram, eye movement and flight data, a quantitative model of the driver three-dimensional space cognitive load is constructed; for specific emergencies, on the basis of analysis of event electroencephalogram related potential components and in combination with driver sight line, the duration of a key instrument is obtained again, and a situational awareness damage index is generated; and fusing the indexes with task management pressure indexes obtained based on the skin and the heart rate to obtain a multi-dimensional cognitive state comprehensive evaluation result, and dynamically adjusting man-machine interaction logic or an automatic driving takeover strategy according to the multi-dimensional cognitive state comprehensive evaluation result. According to the method, the cognitive state of the low-altitude aircraft driver in the complex three-dimensional environment can be objectively and quantitatively evaluated, and technical support is provided for man-machine collaborative optimization and definition of the safety boundary of the low-altitude aircraft.

Description

Technical Field

[0001] This invention relates to the field of human-machine interaction technology for low-altitude aircraft, and in particular to a method for quantitative evaluation of multi-dimensional cognitive states in human-machine collaborative flight for low-altitude aircraft. Background Technology

[0002] With the development of the low-altitude economy, the automation level of low-altitude aircraft, represented by eVTOL and flying cars, is constantly improving, but their human-machine interaction evaluation system is seriously lagging behind. Existing technologies mostly focus on monitoring and warning of cognitive load for general pilots. For example, they analyze EEG and eye-tracking data to identify high, medium, and low load levels and provide warnings of cognitive breakdown points. However, these methods fail to fully consider the unique three-dimensional, highly dynamic, and multi-tasking characteristics of low-altitude flight.

[0003] Specifically, there is a lack of quantitative evaluation tools for key human factors such as three-dimensional spatial cognitive load (e.g., the ability to spatially model altitude and relative position during hovering and obstacle avoidance), psychological stress during vertical takeoff and landing, and the disruption of situational awareness during sudden automated intervention. The limits of pilot capabilities and the takeover thresholds of automated systems lack scientifically defined parameters based on objective physiological data, relying primarily on subjective experience, which constitutes a significant safety hazard. Therefore, there is an urgent need for an objective evaluation method that can delve into specific low-altitude flight scenarios and transform the pilot's internal, invisible cognitive state into quantifiable and analyzable data to optimize human-machine collaboration. Summary of the Invention

[0004] In view of the above, the present invention aims to provide a method for quantitative evaluation of multi-dimensional cognitive state in human-machine cooperative flight for low-altitude aircraft, so as to solve the aforementioned technical problems.

[0005] The technical solution adopted in this invention is as follows:

[0006] This invention provides a multi-dimensional cognitive state quantitative evaluation method for human-machine cooperative flight in low-altitude aircraft, including:

[0007] Simultaneously collect multi-dimensional physiological data of the aircraft pilot and aircraft status data during the flight mission; the multi-dimensional physiological data includes at least electroencephalogram (EEG) signals, eye movement data, skin conductance response, and heart rate variability; the aircraft status data includes at least the aircraft's attitude angle, vertical velocity, and autopilot engagement and disengagement status.

[0008] Based on the power ratio of theta waves to alpha waves in the prefrontal and parietal regions of the EEG signal, and combined with the vertical eye saccade velocity and fixation point distribution, a cognitive load quantification model of the driver in three-dimensional space is constructed, and a three-dimensional spatial cognitive load index is output.

[0009] Based on the preset emergency event, taking the time when the event occurs as zero point, analyzing the latency and amplitude change of the event-related potential component in the brain electrical signal within the preset time window after the event, and synchronously analyzing the time length required for the driver to first gaze at the preset key flight instrument region in the eye movement data, the fusion result of the two indexes is taken as the situation awareness damage index representing the degree of situation awareness break and recovery;

[0010] According to the electrodermal response and heart rate variability, physiological characteristics related to the psychological stress level of the driver are extracted, and a task management stress index is generated;

[0011] The three-dimensional spatial cognitive load index, the situation awareness damage index and the task management stress index are integrated to generate a multi-dimensional cognitive state comprehensive evaluation result for evaluating the current human-machine cooperative flight state of the driver, and the human-machine interaction complexity or the aircraft automatic driving takeover strategy is dynamically adjusted according to the comprehensive evaluation result.

[0012] In at least one possible implementation manner, the manner of constructing the quantitative model of the three-dimensional spatial cognitive load specifically includes:

[0013] The collected brain electrical signals are subjected to time-frequency analysis, the average power of theta waves and alpha waves in the frontal lobe and parietal lobe regions within the task period is extracted, and the ratio θ / α is calculated;

[0014] The eye movement data is synchronously analyzed, the average saccade speed in the vertical direction is extracted, and the gaze point density on a specific height layer or vertical direction obstacle is extracted;

[0015] By associating and coupling the θ / α ratio with the eye movement characteristics in the vertical direction, a comprehensive regression model is established, and the output value of the comprehensive regression model is the three-dimensional spatial cognitive load index.

[0016] In at least one possible implementation manner, the preset emergency event at least includes one of the following: the automatic driving instrument switches from a high-order autonomous flight mode to a low-order mode, or issues an emergency takeover request, or the flight protection mechanism is abnormally triggered.

[0017] In at least one possible implementation manner, the manner of generating the situation awareness damage index specifically includes:

[0018] Taking the occurrence time t0 of the emergency event as zero point, the brain electrical signals from t0 to t0+1000ms are intercepted;

[0019] By superimposing the average algorithm, the peak latency Tp300 and the peak amplitude Ap300 of the P300 component in the event-related potential are identified and measured;

[0020] A time elapsed from the time t0 to when the driver's line of sight first falls on the critical flight instrument region is calculated, and is recorded as an instrument recovery time Treacquire;

[0021] The elongation of Tp300 relative to the baseline, the reduction of Ap300 relative to the baseline, and the Treacquire are weighted and fused to generate the situational awareness impairment index.

[0022] In at least one possible implementation, the human-machine interaction complexity is dynamically adjusted according to the comprehensive evaluation result, and specifically includes: when the three-dimensional space cognitive load index exceeds a preset first threshold, automatically switching a display mode of the three-dimensional navigation map to a two-dimensional profile map, and / or switching obstacle information presented in a symbolic form to a stereoscopic highlight prompt in an augmented reality manner.

[0023] In at least one possible implementation, an automatic driving takeover strategy of the aircraft is dynamically adjusted according to the comprehensive evaluation result, and specifically includes: when the task management pressure index exceeds a preset second threshold, delaying pushing of non-critical system alarm information, and / or increasing a threshold value of automatic execution of a preset safety program by the aircraft.

[0024] Compared with the prior art, the main design concept of the present application is to deeply study the low-altitude flight scene, and a quantitative model for three-dimensional space cognitive load and situational awareness break is proposed to solve the problem of objective evaluation of specific problems in this field. Specifically, the multi-dimensional physiological data of the aircraft pilot during the execution of the flight task and the aircraft state data are synchronously collected; based on the power ratio of the theta wave and the alpha wave in the frontal lobe and the parietal lobe region of the electroencephalogram, and combined with the vertical direction of the eye movement saccade speed and the fixation point distribution, a cognitive load quantitative model of the pilot in the three-dimensional space is constructed, and a three-dimensional space cognitive load index is output; based on the preset emergency event, the event occurrence time is taken as the zero point, the latency and amplitude change of the event-related potential component in the electroencephalogram in the preset time window after the event are analyzed, and the time required for the pilot to first fixate on the preset key flight instrument area in the eye movement data is synchronously analyzed, and the fusion result of the two indexes is taken as the situational awareness damage index representing the degree of situational awareness break and recovery; according to the skin galvanic response and the heart rate variability, the physiological characteristics related to the psychological stress level of the pilot are extracted, and a task management stress index is generated; the three-dimensional space cognitive load index, the situational awareness damage index and the task management stress index are comprehensively integrated to generate a multi-dimensional cognitive state comprehensive evaluation result for evaluating the current human-machine cooperative flight state of the pilot, and the human-machine interaction complexity or the aircraft automatic driving takeover strategy is dynamically adjusted according to the comprehensive evaluation result. The present application not only warns the overload state, but also directly applies the quantitative evaluation result to the dynamic optimization of human-machine interaction, realizes the self-adaptation and intelligentization of human-machine cooperation by automatically adjusting the interface information complexity and the automatic system behavior, and fundamentally prevents the occurrence of cognitive overload and situational break. It can be seen that the present application can provide scientific basis and technical support for formulating industry standards centered on people. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described below in combination with the drawings, in which:

[0026] Figure 1 A schematic diagram of the multi-dimensional cognitive state quantitative evaluation method for human-machine cooperative flight of low-altitude aircraft provided by the embodiment of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application will be described in detail below, and the examples of the embodiments are shown in the drawings, in which the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the present application, and cannot be interpreted as a limitation on the present application.

[0028] The present application proposes an embodiment of a multi-dimensional cognitive state quantitative evaluation method for human-machine cooperative flight of low-altitude aircraft, specifically, Figure 1The method comprises the following steps:

[0029] Step S1, synchronously collecting physiological data of an aircraft pilot and state parameters of the aircraft, wherein the physiological data at least comprises electroencephalogram data, eye movement data, galvanic skin response data and heart rate data;

[0030] In actual construction of the synchronous data collection system, a non-invasive electroencephalogram cap can be worn by the pilot, and electrode points such as frontal lobe Fz, F3, F4 and parietal lobe Pz, P3 and P4 are focused on, an eye movement tracking instrument with vertical direction tracking capability, a galvanic skin response sensor and a heart rate band are provided. At the same time, the aircraft state parameters can be recorded in real time through the aircraft data bus, including but not limited to: pitch angle, roll angle, vertical speed, airspeed, and current mode of the autopilot, such as heading keeping, automatic landing and activation / exit, etc. All the above data streams are marked with a unified system timestamp to ensure millisecond-level synchronization.

[0031] Step S2, combining and analyzing the electroencephalogram data and the eye movement data with the flight state parameters to generate a three-dimensional space cognitive load index;

[0032] Here, a typical low-altitude dense obstacle avoidance flight is taken as an example. The electroencephalogram data of the pilot during task execution is extracted in real time, the signals from the parietal lobe Pz electrode are subjected to short-time Fourier transform, the instantaneous power of the theta wave 4-8 Hz and the alpha wave 8-13 Hz frequency bands is calculated, and the ratio θ / α is obtained.

[0033] At the same time, the eye movement tracking data is analyzed, the average saccade speed Vvertical_saccade of the pilot in the vertical direction is calculated, and the fixation point density Dvertical_fixation (times / second) for the vertical direction obstacle (such as a high-rise building) is calculated, and a comprehensive regression model is constructed:

[0034] L3D=α*(θ / α)+β*(Vvertical_saccade)+γ*(Dvertical_fixation), wherein the weight coefficients α, β and γ are obtained through a large number of flight simulation experiment data in the early stage, and the L3D value output by the model is the three-dimensional space cognitive load index at the current moment.

[0035] Step S3, obtaining the electroencephalogram data and the eye movement data corresponding to the preset moment of the emergency event to generate a situational awareness impairment index;

[0036] In actual operation, a sudden change event can be preset, for example, the autopilot suddenly exits from the automatic cruise mode and is accompanied by a level one takeover request; or a multi-task scene simulating power failure and performing emergency disposal.

[0037] Set the event trigger time as t0, and cut the segment from t0 to t0+1000ms from the EEG data, and superimpose and average the EEG data under multiple similar events to identify the P300 component. The peak latency Tp300 and peak amplitude Ap300 can be measured, and compared with the baseline values (Tbase, Abase) established by the driver in a calm state.

[0038] At the same time, the eye movement data is analyzed synchronously, and the duration from t0 to the first time when the driver's line of sight falls into the preset key instrument area (such as the attitude indicator) is calculated and recorded as Treacquire.

[0039] Finally, the situational awareness impairment index ISA = w1*(Tp300-Tbase)-w2*(Ap300-Abase)+w3*Treacquire is obtained, and those skilled in the art can understand that w1-w3 are predetermined weights, and the higher the ISA index, the more serious the impairment of the driver's situational awareness.

[0040] Step S4, in the emergency, a task management pressure index is generated based on the galvanic skin response data and the heart rate data;

[0041] For example, in a multi-task scenario of simulating power failure and performing emergency disposal, the galvanic skin response signal is analyzed, and the rising slope of skin conductance level ΔGSR is calculated; more preferably, the heart rate variability is analyzed synchronously, and the ratio of low frequency component to high frequency component LF / HF is calculated.

[0042] After normalizing the ratio of average ΔGSRnorm and average LF / HFnorm in the task phase, the task management pressure index in this embodiment is obtained:

[0043] Ptask=(ΔGSRnorm+LF / HFnorm) / 2.

[0044] Step S5, the three-dimensional spatial cognitive load index, the situational awareness impairment index, and the task management pressure index are combined to evaluate the driver's state and perform adaptive interactive optimization.

[0045] In detail, the three indices (L3D, ISA, Ptask) calculated in the previous steps S2, S3, and S4 can be combined to form a three-dimensional evaluation vector to comprehensively describe the current human-machine cooperative flight state of the aircraft driver.

[0046] In actual operation, two threshold values can be preset for the aforementioned 1-3 dimensions: for example, one is Th_L for three-dimensional spatial cognitive load, and the other is Th_P for task management pressure.

[0047] If L3D exceeds Th_L, the three-dimensional terrain map on the primary flight display can be automatically simplified into a two-dimensional profile, and the obstacle information can be superimposed on the head-up display in the form of a highlighted stereoscopic frame.

[0048] If Ptask exceeds Th_P, non-urgent warning information, such as next waypoint distance update, can be automatically delayed in display, and the voice warning playback speed can be slowed down; preferably, the threshold value of the automatic execution of the emergency return procedure of the aircraft can also be increased by 10%, giving the pilot more time for manual handling.

[0049] In summary, the main idea of the present application is to synchronously collect multi-dimensional physiological data of the pilot, such as electroencephalogram, eye movement, skin electricity, heart rate, and flight data of the aircraft; based on the coupling of electroencephalogram, eye movement and flight data, a quantitative model of three-dimensional spatial cognitive load of the pilot is constructed; for a specific emergency event, a situational awareness impairment index is generated based on the analysis of event-related potential components of electroencephalogram and combined with the time length of the pilot regaining sight of the key instrument; the above index and the task management stress index based on skin and heart rate are fused to obtain a comprehensive evaluation result of multi-dimensional cognitive state, and the human-machine interaction logic or automatic driving takeover strategy is dynamically adjusted accordingly. The present application can objectively and quantitatively evaluate the cognitive state of the pilot of the low-altitude aircraft in a complex three-dimensional environment, and provides technical support for human-machine cooperation optimization and definition of the safety boundary of the low-altitude aircraft.

[0050] In the embodiments of the present application, the expressions referring to the orientation are based on the relative concept of the embodiments, and in addition, "at least one" means one or more, and "multiple" means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the cases of A alone, A and B together, and B alone. Wherein A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and rear associated objects. "At least one of the following" and the like means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, wherein a, b, and c can be single or multiple.

[0051] The above detailed description of the structure, features and effects of the present application is based on the embodiments shown in the drawings, but the above is only the preferred embodiment of the present application, and it should be noted that the technical features involved in the above embodiments and preferred modes can be reasonably combined and matched into various equivalent schemes by those skilled in the art without departing from or changing the design idea and technical effects of the present application. Therefore, the present application is not limited to the implementation range shown in the drawings, and any changes or modifications made in accordance with the concept of the present application, or equivalent embodiments with equivalent changes, shall be within the scope of protection of the present application.

Claims

1. A method for quantitatively evaluating multi-dimensional cognitive state of human-machine cooperative flight of low-altitude aircraft, characterized in that, The application relates to a method for evaluating the current human-machine cooperative flight state of a pilot, and belongs to the field of human-machine interaction. The method comprises the following steps: Synchronously collecting multi-dimensional physiological data of a pilot during the execution of a flight task and aircraft state data; The multi-dimensional physiological data at least includes electroencephalogram signals, eye movement data, galvanic skin response and heart rate variability; the aircraft state data at least includes attitude angle, vertical speed and the intervention and exit state of an autopilot; Based on the power ratio of theta waves and alpha waves in the frontal lobe and parietal lobe regions of the electroencephalogram signals, and in combination with the vertical eye movement saccade speed and gaze point distribution, a cognitive load quantification model of the pilot in a three-dimensional space is constructed, and a three-dimensional space cognitive load index is outputted; Based on a preset emergency event, with the time point of the occurrence of the emergency event as zero point, the latency and amplitude variation of an event-related potential component in the electroencephalogram signals within a preset time window after the time point are analyzed, and the time length required by the pilot to first gaze at a preset key flight instrument region in the eye movement data is synchronously analyzed; the fusion result of the two indexes is taken as a situational awareness damage index representing the degree of situational awareness break and recovery; According to the galvanic skin response and heart rate variability, physiological features related to the psychological stress level of the pilot are extracted, and a task management stress index is generated; 2. The method of claim 1, wherein the method is a method of quantitatively evaluating a multi-dimensional cognitive state of a human-machine cooperative flight of a low-altitude aircraft. The three-dimensional space cognitive load index, the situational awareness damage index and the task management stress index are comprehensively combined to generate a multi-dimensional cognitive state comprehensive evaluation result for evaluating the current human-machine cooperative flight state of the pilot, and the human-machine interaction complexity or the autopilot takeover strategy of the aircraft is dynamically adjusted according to the comprehensive evaluation result. The method for constructing the quantification model of the three-dimensional space cognitive load specifically comprises the following steps: Time-frequency analysis is performed on the collected electroencephalogram signals, the average power of theta waves and alpha waves in the frontal lobe and parietal lobe regions within a task period is extracted, and the ratio theta / alpha is calculated; Synchronous analysis is performed on the eye movement data, the average saccade speed in the vertical direction is extracted, and the gaze point density on a specific height layer or vertical direction obstacle is extracted; 3. The method of claim 1, wherein the method is a method of quantitatively evaluating a multi-dimensional cognitive state of a human-machine cooperative flight of a low-altitude aircraft. By coupling the theta / alpha ratio and the eye movement features in the vertical direction, a comprehensive regression model is established, and the output value of the comprehensive regression model is the three-dimensional space cognitive load index.

4. The method of claim 1, wherein the method is a method of quantitatively evaluating a multi-dimensional cognitive state of a human-machine cooperative flight of a low-altitude aircraft. The preset emergency event at least includes one of the following: the autopilot switches from a high-order autonomous flight mode to a low-order mode, or issues an emergency takeover request, or the flight protection mechanism is abnormally triggered. The method for generating the situational awareness damage index specifically comprises the following steps: Taking the time point t0 of the occurrence of the emergency event as zero point, the electroencephalogram signals from t0 to t0+1000 ms are intercepted; By means of superposition average algorithm, the peak latency Tp300 and peak amplitude Ap300 of the P300 component in the event-related potential are identified and measured; The time experienced from the time point t0 to the time when the pilot's line of sight first falls on the key flight instrument region is calculated and recorded as instrument recovery time Treacquire; The elongation of Tp300 relative to the baseline, the reduction of Ap300 relative to the baseline and Treacquire are weighted and fused to generate the situational awareness damage index.

5. The method of claim 1-4, wherein the method is used for the evaluation of the multi-dimensional cognitive state of the human-machine system in low-altitude flight. According to the comprehensive evaluation result, the man-machine interaction complexity is dynamically adjusted, specifically including: when the three-dimensional space cognitive load index exceeds a preset first threshold value, automatically switching the display mode of the three-dimensional navigation map to a two-dimensional profile map, and / or switching the obstacle information presented in a symbolic form to an augmented reality type three-dimensional highlight prompt. ​ 6. The method of claim 1-4, wherein the method is a method for quantitatively evaluating the multi-dimensional cognitive state of human-machine cooperation flight for low-altitude aircraft. According to the comprehensive evaluation result, the aircraft automatic driving takeover strategy is dynamically adjusted, specifically including: when the task management pressure index exceeds a preset second threshold value, delaying the pushing of non-critical system alarm information, and / or increasing the threshold value of the automatic execution of the preset safety program by the aircraft.

Images