Real-time Monitoring System and Method for the Attention State of Low-altitude Controllers Based on EEG Features

Abstract

The present invention discloses a real-time monitoring system and method for the attention state of low-altitude controllers based on EEG features, including an EEG data acquisition and analysis module, a real-time monitoring module for the attention state of controllers, and a real-time monitoring method based on the above modules; it is applied to the monitoring of the attention state of low-altitude controllers during the execution of control tasks, so as to provide support for the design optimization of the human-computer interaction interface, operation process, etc. of controllers in flight service stations or command centers under the low-altitude economy, and ensure the efficiency and safety of low-altitude airspace operation management.

Description

Technical Field

[0001] The present invention belongs to the field of attention state monitoring, and more specifically, relates to a real-time monitoring system and method for the attention state of low-altitude air traffic controllers based on electroencephalogram (EEG) features. Background Art

[0002] With the rapid development of the low-altitude economy and the scale of drone operations, the monitoring tasks of ground air traffic controllers have become increasingly heavy. A good attention state of air traffic controllers can effectively support safe and efficient operations. In particular, the real-time monitoring of the attention state of low-altitude air traffic controllers is of great significance for the optimal design of task operation processes and human-machine interface designs.

[0003] Currently, there are still the following deficiencies in the monitoring of the attention state of low-altitude air traffic controllers:

[0004] Firstly, the current attention state monitoring usually adopts a combined subjective and objective evaluation method, which has a certain degree of subjectivity. Secondly, existing attention state monitoring systems are mostly based on eye movement indicators and lack the exploration of EEG-sensitive features that can reveal deep understanding. Finally, there is currently a lack of development of a real-time monitoring system for air traffic controllers based on EEG features.

[0005] Based on the above situation, a real-time monitoring system and method for the attention state of low-altitude air traffic controllers based on EEG features are designed, which can solve the above-mentioned shortcomings and be applied to the monitoring of the attention state of low-altitude air traffic controllers during the execution of control tasks, thereby providing support for the design optimization of the human-machine interaction interface, operation process, etc. of air traffic controllers in flight service stations or command centers under the low-altitude economy, and ensuring the efficiency and safety of low-altitude airspace operation management. Summary of the Invention

[0006] In view of the above defects, the present invention provides a real-time monitoring system for the attention state of low-altitude air traffic controllers based on EEG features, including:

[0007] an EEG data acquisition and analysis module and a real-time monitoring module for the attention state of air traffic controllers;

[0008] In the EEG data acquisition and analysis module, there are sub-modules for collecting and analyzing EEG features in the attention state of low-altitude air traffic controllers, including a data acquisition sub-module, a data analysis sub-module, and a feature selection sub-module;

[0009] The real-time monitoring module for the attention state of air traffic controllers includes a feature preprocessing sub-module, an attention state discrimination sub-module, and an optimization strategy recommendation sub-module;

[0010] The feature preprocessing sub-module is used to normalize and standardize the EEG features output by the feature selection sub-module in sequence, and the output is the standardized EEG features;

[0011] The attention state discrimination sub-module is used to discriminate the results of no-order attention, primary-secondary order attention, and multi-level attention states output after calculating the EEG features standardized by the feature preprocessing sub-module;

[0012] The optimization strategy recommendation sub-module obtains an optimization strategy corresponding to the discrimination result output by the attention state discrimination sub-module. The optimization strategy includes the optimization of task operations and the optimization of the human-computer interaction interface.

[0013] Further, the calculation process of the attention state discrimination sub-module is as follows:

[0014] Substitute the standardized features into the following equation:

[0015] (1);

[0016] (2);

[0017] (3);

[0018] In equations (1) to (3), P 1, P 2, and P 3 are the discrimination probabilities of no-order attention, primary-secondary order attention, and multi-level attention states respectively, y 1 to y 14 are the standardized EEG features output by the feature preprocessing sub-module, and Ln is the natural logarithm.

[0019] Further, the present invention also discloses a real-time monitoring method for the attention state of low-altitude air traffic controllers based on EEG features, including the above-mentioned real-time monitoring system for the attention state of low-altitude air traffic controllers based on EEG features, and further includes the following steps:

[0020] S1. Collect the EEG data of low-altitude air traffic controllers through the EEG data acquisition and analysis module, and store it in a computer;

[0021] S2. Preprocess and analyze the EEG data collected in step S1 through the data analysis sub-module;

[0022] S2-1. For the EEG data, perform scalp electrode point positioning to obtain the positioned EEG data;

[0023] S2-2. Select A1 and A2 as bilateral mastoid references for the positioned EEG data to obtain the re-referenced EEG data;

[0024] S2-3. Remove the EEG data of useless leads from the re-referenced EEG data to obtain the EEG data of the selected leads;

[0025] S2-4. Perform band-pass filtering on the EEG data of the selected leads in the range of 1 - 30 Hz to obtain the filtered EEG data;

[0026] S2-5. Perform independent component analysis on the filtered EEG data to obtain independent EEG components;

[0027] S2-6. Remove EEG artifacts from the independent EEG components to obtain clean EEG data;

[0028] S2-7. Perform fast Fourier transform on the clean EEG data and divide it into 8 bands;

[0029] S2-8. Calculate the absolute power and relative power of each band;

[0030] S3. Output the absolute power and relative power values of the 8 bands obtained in step S2-8 to the feature selection sub-module for feature selection, and obtain 14 selected EEG features;

[0031] S4. Perform preprocessing on the selected features through the feature preprocessing sub-module;

[0032] S4-1. Normalize the selected features to obtain the normalized features x 1~ x 14 ;

[0033] S4-2. Standardize the above normalization result to obtain the standardized features y 1~ y 14 ;

[0034] S5. Calculate the standardized EEG features through the attention state discrimination sub-module and discriminate the controller's attention state based on the calculation results;

[0035] S6. Recommend optimization strategies for the discrimination results output in step S5 through the optimization strategy recommendation sub-module.

[0036] Further, the 8 bands divided in step S2-7 are specifically: δ (1 - 4 Hz), θ (4 - 8 Hz), α (8 - 12 Hz), σ1(12 - 14 Hz), σ2(14 - 16 Hz), β 1(16 - 20 Hz), β 2(20 - 24 Hz) and β 3(24 - 30 Hz).

[0037] Further, the specific discrimination process in step S5 is:

[0038] For the values obtained in Equations (1) to (3), P 1., P 2 and P 3, compare their magnitudes. When , it is determined that the operator is in a state of disordered attention; when , it is determined that the operator is in a state of primary-secondary order attention; when , it is determined that the operator is in a state of multi-level attention.

[0039] Furthermore, the recommendation of the optimization strategy is specifically as follows:

[0040] When the discrimination result is a state of disordered attention, the optimization strategy displayed on the human-machine interface is:

[0041] a) There is a risk of distraction and loss of situational awareness in the current task operation. Please pay attention to optimizing the workload;

[0042] b) The current human-machine operation interface needs to be optimized;

[0043] When the discrimination result is a state of primary-secondary order attention, the optimization strategy displayed on the human-machine interface is:

[0044] a) There is a risk of high workload in the current task operation. Please verify the design of the emergency response list;

[0045] b) The current human-machine operation interface needs to be optimized. Please check the design of key information or the central area;

[0046] When the discrimination result is a state of multi-level attention, the optimization strategy displayed on the human-machine interface is:

[0047] a) The current task operation is good, and the current workload can be maintained;

[0048] b) The current design of the human-machine operation interface is good.

[0049] The present invention has the following beneficial effects compared with the prior art:

[0050] 1. By collecting and analyzing the EEG signals of air traffic controllers in real time, and further selecting δ _FP2, θ _O1 and other 14 EEG features, the function of real-time discrimination of the attention state of low-altitude air traffic controllers can be achieved;

[0051] 2. For the real-time monitoring method, only by performing analysis steps such as scalp electrode point positioning and re-referencing, the absolute power and relative power features of 8 EEG bands can be calculated. The selected δ _FP2, θBy inputting the discriminant functions of the 14 EEG features such as _O1 into the attention state discrimination sub-module, the attention state discrimination result can be obtained, and further optimization strategies can be given;

[0052] 3. Compared with the traditional attention state monitoring system, the technical solution of the present invention can real-time discriminate the three attention states of air traffic controllers and give optimization strategies. It has the characteristics of good real-time performance and strong sensitivity, and can be used to support the design optimization of the human-computer interaction interface and operation process of air traffic controllers in flight service stations or command centers under the low-altitude economy, so as to ensure the efficiency and safety of low-altitude airspace operation management. Brief Description of the Drawings

[0053] Figure 1 It is a schematic diagram of the real-time monitoring system for the attention state of low-altitude air traffic controllers based on EEG features in the present invention.

[0054] Figure 2 It is the overall working flow chart of the EEG data acquisition and analysis module in the present invention.

[0055] Figure 3 It is the working flow chart of the real-time monitoring module for the attention state of air traffic controllers in the present invention.

[0056] Figure 4 It is the working flow chart of the optimization strategy recommendation sub-module in the present invention.

[0057] Figure 5 It is the initial home page interface diagram of the real-time monitoring system for the attention state of low-altitude air traffic controllers based on EEG features in the present invention.

[0058] Figure 6 It is the interface diagram after selecting the EEG data acquisition and analysis module option under the initial home page interface of the present invention.

[0059] Figure 7 It is the interface diagram after selecting the real-time monitoring module for the attention state of air traffic controllers under the initial home page interface of the present invention. Detailed Description of the Preferred Embodiments

[0060] To facilitate the understanding of the present invention, the device of the present invention will be described more comprehensively below with reference to the relevant drawings. Embodiments of the device are shown in the drawings. However, the device can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0061] In the description of the present invention, it should be noted that unless otherwise clearly specified and limited, the terms "installed", "connected", and "set" should be understood in a broad sense. For example, it can be fixedly connected and set, or detachably connected and set, or integrally connected and set. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0062] Embodiment

[0063] As Figure 1 , this embodiment provides a real-time monitoring system for the attention state of low-altitude controllers based on EEG features, including an EEG data acquisition and analysis module for collecting and analyzing EEG data and selecting EEG features, and a real-time monitoring module for the attention state of controllers for carrying out feature processing, attention state discrimination, and proposing optimization strategies;

[0064] In the EEG data acquisition and analysis module, there is a sub-module for collecting and analyzing EEG features in the attention state of low-altitude controllers, including a data acquisition sub-module for collecting EEG data of low-altitude controllers, a data analysis sub-module for carrying out preprocessing and index analysis of EEG data, and a feature selection sub-module for selecting EEG features;

[0065] Among them, the data analysis of the EEG data output by the data acquisition sub-module by the data analysis sub-module includes the following calculation steps:

[0066] (1) Perform scalp electrode point positioning;

[0067] (2) Convert to bilateral mastoid reference of A1 and A2;

[0068] (3) Remove useless leads;

[0069] (4) 1-30Hz band-pass filtering;

[0070] (5) Independent component analysis;

[0071] (6) Remove artifacts using the Adjust toolbox;

[0072] (7) Divide into 8 bands, namely δ (1-4Hz), θ (4-8Hz), α (8-12Hz), σ1(12-14Hz), σ2(14-16Hz), β 1(16-20Hz), β 2(20-24Hz) and β 3(24-30Hz);

[0073] (8) Calculate the absolute power and relative power of each band;

[0074] The real-time monitoring module for the attention state of controllers includes a feature preprocessing sub-module, an attention state discrimination sub-module, and an optimization strategy recommendation sub-module;

[0075] The feature preprocessing sub-module is used to normalize and standardize the EEG features output by the feature selection sub-module in sequence, and the output is the standardized EEG features. The number of EEG features selected by the feature selection sub-module is 14, which are respectively δ _FP2, that is, the FP2 electrode point δ The absolute power value of the θ _O1, that is, the O1 electrode point θ The absolute power value of the δ _O1, that is, the O1 electrode point δ The absolute power value of the β 1_O1, that is, the O1 electrode point β The absolute power value of the 1 β 3_O1, that is, the O1 electrode point β The absolute power value of the 3 β 2_O1, that is, the O1 electrode point β The absolute power value of the 2 α _O1, that is, the O1 electrode point α The absolute power value of the θ _r_C4, that is, the C4 electrode point θ The relative power value of the δ _CP3, that is, the CP3 electrode point δ The absolute power value of the θ _r_CPZ, that is, the CPZ electrode point θ The relative power value of the θ _r_CZ, that is, the CZ electrode point θ The relative power value of the

[0076] The real-time monitoring module for the controller's attention state includes a supervision feature preprocessing sub-module, an attention state discrimination sub-module, and an optimization strategy recommendation sub-module;

[0077] The supervision feature preprocessing sub-module is used to normalize and standardize the EEG features output by the feature selection sub-module in sequence. The normalized EEG features are defined as x 1~ x 14 And the EEG features after standardization are defined as y 1 to y 14 The specific process is as follows:

[0078] Substitute the standardized features into the following equation:

[0079] (1);

[0080] (2);

[0081] (3);

[0082] In equations (1) to (3), P 1, P 2 and P 3 are the discrimination probabilities of the no-order attention, primary-secondary order attention, and multi-level attention states respectively, y 1 to y 14 are the standardized EEG features output by the feature preprocessing sub-module, and Ln is the natural logarithm;

[0083] The attention state discrimination sub-module is used to input the standardized EEG features and output the discrimination result of the attention state;

[0084] The optimization strategy recommendation sub-module is used to input the discrimination result of the attention state and thus give the corresponding optimization strategy recommendation.

[0085] As Figures 2 - 4 shown, the real-time monitoring method of the real-time monitoring system for the attention state of low-altitude air traffic controllers based on EEG features is as follows:

[0086] S1. Collect the EEG data of the low-altitude air traffic controller through the EEG data acquisition and analysis module and store it in the storage medium inside the computer (the real-time monitoring system for the attention state of low-altitude air traffic controllers based on EEG features is integrated inside the computer, and through Figures 5 - 7 the interface is synchronously displayed), where Figure 6 the data acquisition sub-module of the display page will display the acquisition situation;

[0087] S2. Preprocess and analyze the EEG data collected in step S1 through the data analysis sub-module;

[0088] S2-1. Perform scalp electrode point positioning on the EEG data to obtain the located EEG data;

[0089] S2-2. Select A1 and A2 as the bilateral mastoid references for the located EEG data to obtain the re-referenced EEG data;

[0090] S2-3. Remove the EEG data of the useless leads from the re-referenced EEG data to obtain the EEG data of the selected leads;

[0091] S2-4. Perform band-pass filtering on the EEG data of the selected leads at 1 - 30 Hz to obtain the filtered EEG data;

[0092] S2-5. Perform independent component analysis on the filtered EEG data to obtain independent EEG components;

[0093] S2-6. Remove EEG artifacts from the independent EEG components to obtain clean EEG data;

[0094] S2-7. Perform fast Fourier transform on the clean EEG data and divide it into 8 bands, namely δ (1 - 4 Hz), θ (4 - 8 Hz), α (8 - 12 Hz), σ1(12 - 14 Hz), σ2(14 - 16 Hz), β 1(16 - 20 Hz), β 2(20 - 24 Hz) and β 3(24 - 30 Hz);

[0095] S2-8. Calculate the absolute power and relative power of each band;

[0096] S3. Output the absolute power and relative power values of the 8 bands obtained in step S2-8 to the feature selection sub-module for feature selection, and obtain 14 selected EEG features;

[0097] S4. Perform preprocessing on the selected features through the feature preprocessing sub-module;

[0098] S4-1. Normalize the selected features to obtain the normalized features x 1 to x 14 ;

[0099] S4-2. Standardize the above normalization result to obtain the standardized features y 1 to y 14 ;

[0100] S5. Calculate the standardized EEG features through the attention state discrimination sub-module and discriminate the controller's attention state from the calculation results. Specifically:

[0101] Compare the magnitudes of the P 1, P 2 and P 3 values obtained in equations (1) to (3) of the supervision feature preprocessing sub-module. When , it is determined that the operator is in a non-sequential attention state; when , it is determined that the operator is in a primary-secondary sequential attention state; when , it is determined that the operator is in a multi-level attention state;

[0102] S6. Recommend optimization strategies for the discrimination result output in step S5 through the optimization strategy recommendation sub-module, as shown in the operation interface of Figure 7 as follows:

[0103] When the discrimination result is the unordered attention state, the optimization strategies displayed on the human-machine interaction interface are:

[0104] a) There are risks of distraction and loss of situational awareness in the current task operation. Please pay attention to optimizing the workload;

[0105] b) The current human-machine operation interface needs to be optimized;

[0106] When the discrimination result is the primary-secondary order attention state, the optimization strategies displayed on the human-machine interaction interface are:

[0107] a) There is a risk of high load in the current task operation. Please verify the design of the emergency response checklist;

[0108] b) The current human-machine operation interface needs to be optimized. Please check the design of key information or the central area;

[0109] When the discrimination result is the multi-level attention state, the optimization strategies displayed on the human-machine interaction interface are:

[0110] a) The current task operation is good and the current workload can be maintained;

[0111] b) The current design of the human-machine operation interface is good.

[0112] It should be noted that the structure described in the present invention can be implemented in many different forms and is not limited to the embodiments. Any equivalent transformation made by those of ordinary skill in the art using the description and drawings of the present invention, or directly or indirectly applied in other related technical fields, such as the loading and unloading of other items, is included in the protection scope of the present invention.

Claims

1. A real-time monitoring system for the attention state of low-altitude air traffic controllers based on electroencephalogram features, characterized in that, Including: An electroencephalogram (EEG) data acquisition and analysis module and a real-time monitoring module for the attention state of air traffic controllers; In the EEG data acquisition and analysis module, there is a sub-module for collecting and analyzing EEG characteristics in the attention state of low-altitude air traffic controllers, including a data acquisition sub-module, a data analysis sub-module, and a feature selection sub-module; Among them, the data analysis of the EEG data output by the data acquisition sub-module in the data analysis sub-module includes the following calculation steps: 1) Perform scalp electrode point positioning; 2) Convert to bilateral mastoid reference of A1 and A2; 3) Remove useless leads; 4) 1 - 30 Hz band-pass filtering; 5) Independent component analysis; 6) Remove artifacts using the Adjust toolbox; 7) By Fourier transform, divide into 8 bands, namely δ (1 - 4 Hz), θ (4 - 8 Hz), α (8 - 12 Hz), σ1(12 - 14 Hz), σ2(14 - 16 Hz), β 1(16 - 20 Hz), β 2(20 - 24 Hz) and β 3(24 - 30 Hz); 8) Calculate the absolute power and relative power of each frequency band; The real-time monitoring module for the attention state of air traffic controllers includes a feature preprocessing sub-module, an attention state discrimination sub-module, and an optimization strategy recommendation sub-module; The feature preprocessing sub-module is used to perform normalization and standardization on the EEG features output by the feature selection sub-module in sequence, and the output is 14 standardized EEG features, which are respectively δ _FP2, i.e., the FP2 electrode point δ The absolute power value of the θ _O1, i.e., the O1 electrode point θ The absolute power value of the δ _O1, i.e., the O1 electrode point δ The absolute power value of the β 1_O1, i.e., the O1 electrode point β The absolute power value of the 1 β 3_O1, i.e., the O1 electrode point β The absolute power value of the 3 β 2_O1, i.e., the O1 electrode point β The absolute power value of the 2 α _O1, i.e., the O1 electrode point α The absolute power value of the θ _r_C4, i.e., the C4 electrode point θ The relative power value of the δ _CP3, i.e., the CP3 electrode point δ The absolute power value of the θ _r_CPZ, i.e., the CPZ electrode point θ The relative power value of the θ _r_CZ, i.e., the CZ electrode point θ The relative power value of the σ2_r_FT8, i.e., the relative power value of the σ2 band of the FT8 electrode point; The attention state discrimination sub-module is used to discriminate three attention states: unordered attention, primary-secondary order attention, and multi-level attention state, after calculating the 14 EEG characteristics standardized by the feature preprocessing sub-module; The optimization strategy recommendation sub-module obtains corresponding optimization strategies for the discrimination results output by the attention state discrimination sub-module. The optimization strategies include the optimization of task operations and the optimization of the human-computer interaction interface.

2. The real-time monitoring system for the attention state of low-altitude controllers based on EEG features according to claim 1, characterized in that, The calculation process of the attention state discrimination sub-module is as follows: Substitute the standardized features into the following equation: （1）； （2）； （3）； In equations (1) to (3), P 1, P 2, and P 3 are the discrimination probabilities of the non-sequential attention, primary-secondary sequential attention, and multi-level attention states, respectively, y 1 to y 14 are the standardized EEG features output by the feature preprocessing sub-module, and Ln is the natural logarithm.

3. A real-time monitoring method for the attention state of low-altitude controllers based on EEG features, characterized in that: The real-time monitoring system for the attention state of low-altitude air traffic controllers according to any one of claims 1 to 2 based on EEG characteristics further includes the following steps: S1. Collect the EEG data of low-altitude air traffic controllers through the EEG data acquisition and analysis module and store it in a computer; S2. Preprocess and analyze the EEG data collected in step S1 through the data analysis sub-module; S2-1. For the EEG data, perform scalp electrode point positioning to obtain the positioned EEG data; S2-2. Select A1 and A2 as bilateral mastoid references for the positioned EEG data to obtain the re-referenced EEG data; S2-3. For the re-referenced EEG data, remove the EEG data of useless leads to obtain the EEG data of the selected leads; S2-4. Perform 1 - 30 Hz band-pass filtering on the EEG data of the selected leads to obtain the filtered EEG data; S2-5. Perform independent component analysis on the filtered EEG data to obtain independent EEG components; S2-6. Remove EEG artifacts from the independent EEG components to obtain clean EEG data; S2-7. Perform fast Fourier transform on the clean EEG data and divide it into 8 frequency bands; S2-8. Calculate the absolute power and relative power of each frequency band; S3. Output the absolute power and relative power values of the 8 frequency bands obtained in step S2-8 to the feature selection sub-module for feature selection to obtain 14 selected EEG characteristics; S4. Preprocess the selected features through the feature preprocessing sub-module; S4-1. Normalize the selected features to obtain the normalized features x 1~ x 14 ; S4-2. Standardize the above normalization result to obtain the standardized features y 1~ y 14 ; S5. Calculate the standardized EEG characteristics through the attention state discrimination sub-module and discriminate the attention state of air traffic controllers for the calculation results; S6. Recommend optimization strategies for the discrimination results output in step S5 through the optimization strategy recommendation sub-module.

4. The real-time monitoring method for the attention state of low-altitude controllers based on EEG features according to claim 3, wherein The specific discrimination process in step S5 is as follows: For the values obtained in Equations (1) to (3), P 1, P 2, and P 3 are compared. When , it is determined that the operator is in a state of disordered attention; when , it is determined that the operator is in a state of primary-secondary order attention; when , it is determined that the operator is in a state of multi-level attention.

5. The real-time monitoring method for the attention state of low-altitude controllers based on EEG features according to claim 4, characterized in that The specific recommendation of the optimization strategy is as follows: When the discrimination result is the unordered attention state, the optimization strategies displayed on the human-machine interaction interface are: a) There are risks of distraction and loss of situational awareness in the current task operation. Please pay attention to optimizing the workload. b) The current human-machine operation interface needs to be optimized. When the discrimination result is the primary-secondary order attention state, the optimization strategies displayed on the human-machine interaction interface are: a) There is a risk of high load in the current task operation. Please verify the design of the emergency response list. b) The current human-machine operation interface needs to be optimized. Please check the design of key information or the central area. When the discrimination result is the multi-level attention state, the optimization strategies displayed on the human-machine interaction interface are: a) The current task operation is good, and the current workload can be maintained. b) The current human-machine operation interface is well-designed.

Images