Allarity quantification method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an alertness quantification method and device, equipment and a storage medium, and the method comprises the steps: carrying out the feature extraction of obtained target electroencephalogram signal data of a to-be-monitored person, and obtaining an event-related potential feature and a power ratio feature; then, on the basis of a pre-constructed alertness quantitative model, performing state judgment on amplitudes and latency periods of the P300 feature and the N200 feature to obtain a state value; performing intensity judgment on the state value based on an alertness quantitative model to obtain a signal intensity value; and finally, through an alertness quantification model, performing alertness quantification on the signal intensity value and the power ratio characteristic to obtain an alertness quantification result. Therefore, according to the method and the device, through fuzzy reasoning of the multiple feature inputs related to the alertness decline effect, the mapping relation with the clear alertness quantification level is determined, unexpected accidents caused by inaccurate alertness quantification levels of the to-be-monitored personnel are avoided, and the accuracy and the precision of alertness quantification are improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of alertness assessment, and relate to but are not limited to an alertness quantification method, apparatus, device, and storage medium. Background Art

[0002] As the receiver, processor and decision maker of traffic information, the driver's driving behavior directly affects and determines driving safety. Long-term fatigue driving will lead to physiological and psychological function decline, resulting in slow reaction and slow movement of the driver, as well as reduced ability to perceive, judge and control vehicles and aircraft in the external environment. In this case, the probability of an accident or near-accident is 4 to 6 times that of an awake state, which poses a great threat to driving safety. Vigilance refers to the ability to process and maintain self-directed awake processing of repetitive, non-awakening stimuli, and the vigilance of personnel will change with the increase of task time, such as slower reaction time and lower accuracy, which is the vigilance decrement effect. How to accurately monitor the vigilance state of personnel in real time and accurately quantify the low vigilance state is one of the current research hotspots.

[0003] In recent years, the quantification and classification methods of alertness based on EEG signals have received extensive attention. Different EEG decoding algorithms are used to extract robust features for classification or regression to obtain alertness levels, which has the advantages of high real-time performance and strong reliability. For example, the classification method based on support vector machine and particle swarm optimization, the classification method of fusion of EEG features in time-frequency domain, etc.

[0004] However, the existing technologies for monitoring alertness status have the following problems: (1) The existing alertness classification model, whose true classification label mainly uses the threshold of the reaction time of the subjective questionnaire scale to evaluate information or behavioral performance of the monitored person, cannot well reflect the effect of decreased alertness as the task time increases; (2) The existing classification model only performs binary classification judgment on the alertness status, and there is still a lack of research on the classification of alertness status; (3) The existing deep learning-based alertness classification model has poor interpretability. Summary of the invention

[0005] Based on the problems in the related art, the embodiments of the present application provide a method, device, equipment and storage medium for quantifying alertness.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] The present application provides a method for quantifying alertness, the method comprising:

[0008] Obtaining target EEG signal data of the person to be monitored;

[0009] Extracting features of the target EEG signal data to obtain event-related potential features and power ratio features; the event-related potential features at least include P300 features and N200 features;

[0010] Based on a pre-built alertness quantification model, the amplitude and latency of the P300 feature and the N200 feature are judged to obtain a state value;

[0011] Based on the alertness quantification model, performing strength judgment on the state value to obtain a signal strength value;

[0012] Based on the alertness quantification model, the signal strength value and the power ratio feature are quantified to obtain an alertness quantification result of the person to be monitored.

[0013] The present application provides a device for quantifying alertness, the device comprising:

[0014] An acquisition module, used to obtain target EEG signal data of the person to be monitored;

[0015] An extraction module, used for performing feature extraction on the target EEG signal data to obtain event-related potential features and power ratio features; the event-related potential features at least include P300 features and N200 features;

[0016] A judgment module, used to make a state judgment on the amplitude and latency of the P300 feature and the N200 feature based on a pre-built alertness quantification model to obtain a state value;

[0017] The judgment module is further used to perform strength judgment on the state value based on the alertness quantification model to obtain a signal strength value;

[0018] The quantification module is used to perform alertness quantification on the signal strength value and the power ratio feature based on the alertness quantification model to obtain the alertness quantification result of the person to be monitored.

[0019] In some embodiments, the acquisition module is also used to obtain the initial EEG signal data of the person to be tested; the initial EEG signal data is processed in sequence by average reference, low-frequency filtering, artifact removal, data segmentation and baseline correction to obtain the target EEG signal data.

[0020] In some embodiments, the extraction module is further used to divide the target EEG signal data into multiple frequency bands; and use a preset power spectrum density estimation method to calculate the multiple frequency bands to obtain the power ratio feature.

[0021] In some embodiments, the alertness quantification model includes a first layer of fuzzy logic; the judgment module is also used to input the amplitude and latency of the P300 feature and the N200 feature into the first layer of fuzzy logic for state judgment to obtain the state value.

[0022] In some embodiments, the alertness quantification model includes a second layer of fuzzy logic; the judgment module is further used to input the state value into the second layer of fuzzy logic to perform strength judgment to obtain the signal strength value.

[0023] In some embodiments, the alertness quantification model includes a third layer of fuzzy logic; the judgment module is further used to input the state value into the third layer of fuzzy logic to perform alertness quantification to obtain the alertness quantification result.

[0024] An embodiment of the present application provides an alertness quantification device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned alertness quantification method when executing the executable instructions stored in the memory.

[0025] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned alertness quantification method.

[0026] The alertness quantification method, device, equipment and storage medium provided in the embodiment of the present application extracts features from the target EEG signal data of the monitored person to obtain event-related potential features and power ratio features; then, based on the pre-constructed alertness quantification model, the amplitude and latency of the P300 feature and the N200 feature are judged to obtain the state value; then, based on the alertness quantification model, the state value is judged to obtain the signal strength value; finally, the signal strength value and the power ratio feature are quantified by the alertness quantification model to obtain the alertness quantification result. In this way, the present application determines a mapping relationship with clear alertness quantification levels by combining reasoning such as fuzzification, fuzzy reasoning, fuzzy rules, and defuzzification of multiple features related to the alertness reduction effect (such as the amplitude and latency of P300 and N200, and the power ratio), thereby improving the accuracy and precision of quantifying the alertness of the monitored person, and effectively improving safety and security in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a structural diagram of a system for quantifying alertness provided in an embodiment of the present application;

[0028] Figure 2 It is a flowchart of a method for quantifying alertness provided in an embodiment of the present application;

[0029] Figure 3 It is a schematic diagram of the Mackworth watch test provided in the embodiment of the present application;

[0030] Figure 4 It is a schematic diagram of the process of preprocessing EEG signals provided in an embodiment of the present application;

[0031] Figure 5 Schematic diagram of P300 waveforms obtained from six time windows of four experiments provided in the embodiment of the present application;

[0032] Figure 6 Schematic diagram of N200 waveforms obtained from six time windows of four experiments provided in the embodiments of the present application;

[0033] Figure 7 It is a schematic diagram of a vigilance quantification model based on three-layer fuzzy logic provided in an embodiment of the present application;

[0034] Figure 8 is a graph showing the average reaction time versus time provided in the embodiments of the present application;

[0035] Fig. 9 A schematic diagram of the structure of the alertness quantification device provided in an embodiment of the present application;

[0036] Fig.10 A schematic diagram of the composition structure of the alertness quantification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0038] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0039] The following describes an exemplary application of the alertness quantification device of the embodiment of the present application. The alertness quantification device provided in the embodiment of the present application can be implemented as a terminal or as a server. In one implementation, the alertness quantification device provided in the embodiment of the present application can be implemented as various types of terminals such as laptops, tablet computers, desktop computers, mobile devices, etc.; in another implementation, the alertness quantification device provided in the embodiment of the present application can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiment of the present application. Below, an exemplary application of the alertness quantification device when it is implemented as a server will be described.

[0040] join Figure 1 , Figure 1 : is a structural diagram of the alertness quantification system provided in the embodiment of the present application. In order to realize the alertness quantification of the monitored personnel, the embodiment of the present application can provide an alertness quantification platform, and the alertness quantification platform can be implemented as an alertness quantification application. The alertness quantification system 10 provided in the embodiment of the present application includes a terminal 110, a network 120 and a server 130, wherein the server 130 is a server of the alertness quantification application. The server 130 can constitute the alertness quantification device of the embodiment of the present application. The terminal 110 is connected to the server 130 via the network 120, and the network 200 can be a wide area network or a local area network, or a combination of the two.

[0041] In some embodiments, please refer to Figure 1 When performing alertness quantification, the terminal 110 sends the target EEG signal data to the server 130 through the network 120. The server 130 extracts features from the target EEG signal data to obtain event-related potential features and power ratio features; the event-related potential features include at least P300 features and N200 features; then, based on the pre-built alertness quantification model, the amplitude and latency of the P300 feature and the N200 feature are judged to obtain the state value; based on the alertness quantification model, the state value is judged to obtain the signal strength value; based on the alertness quantification model, the signal strength value and the power ratio feature are alertly quantified to obtain the alertness quantification result of the person to be monitored. After obtaining the alertness quantification result, the server 130 sends the alertness quantification result to the terminal 110 through the network 120.

[0042] In some embodiments, the terminal 110 can also implement the alertness quantification method, that is, the terminal acts as the executor to extract features from the target EEG signal data corresponding to the monitored person to obtain event-related potential features and power ratio features; the event-related potential features include at least P300 features and N200 features; then, based on a pre-constructed alertness quantification model, the amplitude and latency of the P300 feature and the N200 feature are judged to obtain the state value; based on the alertness quantification model, the state value is judged to obtain the signal strength value; finally, based on the alertness quantification model, the signal strength value and the power ratio feature are alertness quantified to obtain the alertness quantification result.

[0043] The alertness quantification method provided in the embodiment of the present application can also be implemented based on a cloud platform and through cloud technology. For example, the server 130 can be a cloud server. The target EEG signal data is feature extracted through the cloud server, or the amplitude and latency of the P300 feature and the N200 feature are judged through the cloud server, or the state value is judged through the cloud server, or the signal strength value and power ratio feature are quantified through the cloud server.

[0044] In some embodiments, a cloud storage may be provided, and the target EEG signal data may be stored in the cloud storage, or a pre-built alertness quantification model may be stored in the cloud storage. When the server performs alertness quantification on the target EEG signal data, the target EEG signal data may be accurately and efficiently quantified based on the stored alertness quantification model to obtain an alertness quantification result.

[0045] It should be noted here that cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0046] This application embodiment provides a method for quantifying alertness. Figure 2 , Figure 2 is a flowchart of a method for quantifying alertness provided in an embodiment of the present application, which will be combined with Figure 2 The steps shown are explained.

[0047] Step S210, obtaining target EEG signal data of the person to be monitored.

[0048] Here, the persons to be monitored refer to those who need to have their alertness quantified. The persons to be monitored may be drivers, pilots, navigators, duty personnel, etc.

[0049] In some embodiments, the target EEG signal data refers to the data obtained after data preprocessing of the collected EEG data. Here, the data preprocessing includes average reference, 1-45Hz low-frequency bandpass filtering, artifact removal (±110μV as artifact threshold), segmentation of the data of [-200ms, 500ms] (200ms before the target appears to 500ms after the target appears), baseline correction of the data of [-200ms, 0ms] (0ms indicates the time point when the target appears), etc. The target EEG signal data obtained is used for subsequent analysis and processing.

[0050] Step S220, extracting features from the target EEG signal data to obtain event-related potential features and power ratio features.

[0051] In some embodiments, event-related potential (ERP) features are used to characterize the activity patterns of the brain of the monitored person under different alertness states.

[0052] In some embodiments, the power ratio feature refers to the ratio between the powers of EEG signals in different frequency bands, which reflects the relative intensity of brain activity in different frequency bands. By calculating the power ratio between these frequency bands, the brain's alertness level can be evaluated. In practical applications, event-related potential features include P300 potential features and N200 potential features. By analyzing the latency, amplitude, and waveform characteristics of these features, the alertness level of the person to be monitored can be evaluated.

[0053] In practical applications, feature extraction can be performed in any manner, and this embodiment of the present application does not specifically limit this.

[0054] Step S230, based on the pre-built alertness quantification model, the amplitude and latency of the P300 feature and the N200 feature are judged to obtain a state value.

[0055] In some embodiments, the alertness quantification model refers to a model for quantitatively calculating the alertness of the monitored personnel. The alertness quantification model can be a model based on a neural network, a model based on fuzzy logic, or a model based on an attention mechanism. This embodiment of the application does not specifically limit this.

[0056] In an embodiment of the present application, the alertness quantification model is an alertness quantification model based on three-layer fuzzy logic, which determines a clear mapping relationship of alertness quantification through combined reasoning including fuzzification, fuzzy rules, defuzzification, etc. of multiple electrophysiological feature inputs related to the alertness reduction effect (such as the amplitude and latency of the P300 potential and the N200 potential, and the power ratio). In the first layer of fuzzy logic of the model, the input signals are the amplitude and latency of P300 and N200, and the outputs are four types of states of P300 and N200 (no P300 (NP), low P300 (LP), medium P300 (MP), high P300 (HP), no N200 (NN), low N200 (LN), medium N200 (MN), high N200 (HN)); in the second layer of fuzzy logic of the model, the above-mentioned states of P300 and N200 are input, and the outputs are four types of intensity values ​​of event-related potential signals (no event-related potential (NE), low event-related potential (LE), medium event-related potential (ME), high event-related potential (HE)); in the third layer of fuzzy logic of the model, the input signals are signal intensity values ​​and extracted power ratio features, and the final outputs are three types of quantitative levels of alertness, including: high alertness (HV), medium alertness (MV), and fatigue (LV).

[0057] In some embodiments, the amplitude refers to the maximum voltage of the event-related potential collected when a neuron group discharges in a specific brain area, which is an important indicator for measuring the intensity of the event-related potential.

[0058] In some embodiments, latency refers to the time interval from the onset of target stimulation to the appearance of event-related potential, which reflects the speed at which the brain processes the stimulation.

[0059] Step S240: Based on the alertness quantification model, the strength of the state value is judged to obtain a signal strength value.

[0060] In some embodiments, the signal strength value refers to a numerical value used to characterize the strength of the status value.

[0061] Step S250: Based on the alertness quantification model, the signal strength value and the power ratio feature are quantified to obtain an alertness quantification result of the person to be monitored.

[0062] In the present application, the alertness quantification result may include three levels of alertness, namely, high alertness, medium alertness, and fatigue.

[0063] In some embodiments, alertness quantification may be performed based on the power ratio feature obtained in step S220 and the signal strength value obtained in step S240 to obtain an alertness quantification result.

[0064] The alertness quantification method, device, equipment and storage medium provided in the embodiment of the present application extracts features from the target EEG signal data of the monitored person to obtain event-related potential features and power ratio features; then, based on the pre-constructed alertness quantification model, the amplitude and latency of the P300 feature and the N200 feature are judged to obtain the state value; then, based on the alertness quantification model, the state value is judged to obtain the signal strength value; finally, the signal strength value and the power ratio feature are quantified through the alertness quantification model to obtain the alertness quantification result. In this way, the present application determines a mapping relationship with clear alertness quantification levels by combining reasoning such as fuzzification, fuzzy rules, and defuzzification of multiple features related to the alertness reduction effect (such as the amplitude and latency of P300 and N200, and the power ratio), thereby improving the accuracy and precision of alertness quantification and effectively improving safety and security in practical applications.

[0065] In some embodiments, when obtaining the target EEG signal data of the person to be monitored, the collected initial EEG signal may be preprocessed to ensure a clean and noise-free EEG signal, thereby ensuring the accuracy of the alertness quantification. Based on the above embodiment, the above step S210 may be implemented by the following steps S211 to S212:

[0066] Step S211: the server obtains the initial EEG signal data of the person to be detected.

[0067] In the embodiment of the present application, the user can send the initial EEG signal data collected by the EEG signal collector to the server through the network, or upload the initial EEG signal data collected by the EEG signal collector to the server through the hardware interface.

[0068] In step S212, the server sequentially performs average reference, low-frequency filtering, artifact removal, data segmentation and baseline correction processing on the initial EEG signal data to obtain the target EEG signal data.

[0069] In some embodiments, after acquiring the initial EEG signal data, the server may pre-process the initial EEG signal data. During the pre-processing, the target EEG signal data may be obtained through average reference, low-frequency filtering, artifact removal, data segmentation and baseline correction.

[0070] In some embodiments, the above step S220 may be implemented through steps S221 to S222:

[0071] Step S221, dividing the target EEG signal data into multiple frequency bands.

[0072] In some embodiments, the preprocessed EEG signal data can be divided into four frequency bands, namely, δ band (0.1-4 Hz), θ band (4-8 Hz), α band (8-13 Hz), and β band (14-45 Hz).

[0073] Step S222: Using a preset power spectrum density estimation method, the multiple frequency bands are calculated to obtain a power value of each frequency band, and then the power ratio feature is obtained.

[0074] In some embodiments, the power in these fixed frequency bands may be calculated using the Pwelch method.

[0075] In some embodiments, the alertness quantification model includes a first layer of fuzzy logic; the above step S230 can be implemented in the following manner:

[0076] The amplitudes and latency of the P300 feature and the N200 feature are respectively input into the first layer of fuzzy logic for state judgment to obtain the state value.

[0077] In some embodiments, the alertness quantification model includes a second layer of fuzzy logic; the above step S240 can be implemented in the following manner:

[0078] The state value is input into the second layer fuzzy logic to perform strength judgment to obtain the signal strength value.

[0079] In some embodiments, the alertness quantification model includes a third layer of fuzzy logic; the above step S250 can be implemented in the following manner:

[0080] The state value is input into the third layer fuzzy logic to perform alertness quantification to obtain the alertness quantification result.

[0081] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0082] In this application, fuzzy logic refers to solving practical problems by using the experience and reasoning accumulated by the designer, and fuzzy rules are a series of rules designed based on experience or simulation experiments to achieve characteristic effects. Different from the classical binary logic, where 0 represents "false" and 1 represents "true", the propositions in fuzzy logic are no longer either true or false. It can be considered as "partially true" or "partially false". Based on this, fuzzy logic cancels the either-or opposition between binary values, uses membership to represent the transition state between binary values, and maps the input space to the closed interval transition state of the non-binary space. Therefore, this feature can be used to achieve the three classifications of high alertness, medium alertness, and fatigue. Fuzzy logic starts from the macroscopic point of view and grasps the fuzzy characteristics of human brain thinking. It can imitate people's comprehensive inference (experience, feeling, understanding) to deal with fuzzy information processing problems that are difficult to solve with conventional mathematical methods. Therefore, this feature can be used to determine the mapping relationship of alertness using event-related potential characteristics and power spectrum characteristics.

[0083] Fuzzy logic does not rely on the mathematical model of the controlled object, but only on the experience and knowledge of experts; it has good robustness and adaptability, can effectively control complex systems, and can solve nonlinear problems well. Because this method imitates human comprehensive judgment, it has strong model interpretability.

[0084] In this application, based on the theory of the vigilance reduction effect, an objective division of alertness and fatigue states is achieved in the time domain for the fatigue state induced after a long task; a vigilance quantification method based on three-layer fuzzy logic is proposed: through the combined reasoning of fuzzification, fuzzy rules, defuzzification, etc. of multiple electrophysiological feature inputs related to the vigilance reduction effect (such as the amplitude and latency of P300 and N200, power ratio), a clear mapping relationship for vigilance quantification is determined.

[0085] In some embodiments, long-duration tasks can induce a decrease in the brain's alertness, i.e., a fatigue state. Taking the 30-minute Mackworth clock experiment as an example, the behavioral data is first statistically analyzed to observe changes in reaction time, accuracy, false alarm rate, etc. in the time domain to verify the actual performance of the decreased alertness effect in a specific experiment. Different from the currently commonly used method of using the subject's subjective self-evaluation in the experiment as a label to classify alertness, the alertness and fatigue states are objectively divided in the time domain. Taking the 30-minute Mackworth clock experiment as an example, six 5-minute time windows are divided in the time domain, with the first time window as alertness and the sixth time window as fatigue, and this division strategy is used to judge the final output of the alertness quantification.

[0086] The present application proposes a method for quantifying alertness based on three-layer fuzzy logic. In the first layer of fuzzy logic, the input signals are the amplitude and latency of P300 and N200, and the outputs are four types of states (none, low, medium, and high); in the second layer of fuzzy logic, the input signals are the states of P300 and N200, and the outputs are four types of intensity values ​​of event-related potential signals (none, low, medium, and high); in the third layer of fuzzy logic, the input signals are the intensity value and power ratio of event-related potentials, and the final outputs are three types of quantitative values ​​of alertness (high alertness, medium alertness, and fatigue).

[0087] 1) Alertness EEG signal data collection

[0088] In this application, a total of 11 subjects participated in the experiment, including 9 males and 2 females. All subjects were right-handed, healthy, had no history of brain trauma, normal naked eye vision or corrected vision, no abnormal symptoms of the central nervous system, and did not take psychotropic drugs. All experimental procedures were approved by the Experimental Animal Ethics Committee. Before the experiment began, all subjects participating in this experiment read and signed the written informed consent form in advance.

[0089] The experiment used NE's Starstim TM 20-channel EEG cap, device sampling rate 500Hz, reference electrode CMS, analog-to-digital conversion rate 24bit, powered by lithium battery, single battery power supply time up to 4h. The experiment was completed in a shielded room without noise interference. During the entire experiment, the subjects sat on a chair with armrests at a suitable height and distance from the screen, and were required to concentrate and conduct the experiment according to the prompts of the paradigm. In addition, in order to ensure that the data contains as few artifacts as possible such as electromyography, eye movement, and electrooculography, except for the necessary keyboard operations, the subjects were required to keep their heads and bodies as still as possible during data collection, and to reduce swallowing, chewing and other movements as much as possible. The subjects need to complete 4 30-minute MCT experiments.

[0090] The EEG cap complies with the international standard 10-20 system. This application selects 18 electrodes (P3, P4, P7, P8, T7, T8, F4, F7, F8, C3, C4, Fp1, Fz, Cz, Pz, O1, Oz, O2) to collect the EEG data of the subjects, and ensures that the impedance of all electrodes is reduced to below 10kΩ.

[0091] This application selects the Mackworth Clock Test (MCT) as the paradigm for inducing decreased vigilance. The paradigm is written based on the Psychtoolbox platform of MATLAB. First, the subjects are shown a visual display with 30 hole-shaped white circles on a black background. These circles are arranged in a clock-shaped circular figure with a radius of 20 cm. The subjects need to keep their eyes on the center of the clock as much as possible. During the experiment, the green circle jumps once every 1 second. When the green circle jumps to the next circle, it is regarded as a normal signal. At this time, the subjects do not need to do anything; when the green circle moves to twice the usual distance (that is, skipping a circle), it is considered a critical signal. At this time, the participants need to press the space bar on the keyboard as quickly as possible with their fingers to respond to this signal. The experiment lasts for 30 minutes. The experiment is carried out after 5 minutes of practice before the formal experiment. The critical signal appears 3-5 times in each circle. About 60 circles can jump in 30 minutes, and the critical signal appears 180-300 times. The paradigm display interface is as follows Figure 3 shown.

[0092] 2) EEG signal data preprocessing

[0093] In some embodiments, in order to obtain clean and noise-free EEG signals, the collected EEG data are preprocessed, mainly including average reference, 1-45Hz low-frequency bandpass filtering, artifact removal (±110μV as artifact threshold), segmentation of the data of [-200ms, 500ms] (200ms before the target appears to 500ms after the target appears), baseline correction of the data of [-200ms, 0ms] (0ms indicates the time point when the target appears), etc., as follows Figure 4 As shown, the obtained target EEG signal data is used for subsequent analysis and processing.

[0094] 3) Alert EEG feature extraction

[0095] 1) Event-related potential characteristics: P300 mainly appears in the leads of the central area and parietal lobe (C3, C4, Cz, P3, P4, Pz), and N200 mainly appears in the leads of the occipital lobe and parietal lobe (O1, O2, P3, P4, P7, P8). After averaging all subjects and the corresponding lead sets, the P300 waveform and N200 waveform are obtained. Here, the amplitudes of P300 and N200 are defined as the average amplitudes of 20ms before and after the highest peak and the lowest peak, respectively, and the corresponding latency is defined as the time interval from the appearance of the target to the appearance of the peak. Finally, the average amplitude and average latency of P300 and N200 in each 5-minute time window are obtained, as shown below: Figure 5 and 6 shown.

[0096] 2) Power ratio feature: The preprocessed EEG data is divided into four frequency bands, namely, the δ band (0.1-4Hz), the θ band (4-8Hz), the α band (8-13Hz), and the β band (14-45Hz). The power of these fixed frequency bands is calculated using the Pwelch method:

[0097]

[0098] In this application, the (α+θ) / β power ratio feature, which has been shown to be related to the alert state, was selected and the average power ratio feature within each 5-minute time window was calculated.

[0099] IV) Construction method of alertness model based on three-layer fuzzy logic

[0100] 1) Fuzzification

[0101] Fuzzification is the process of converting the concept of real things into fuzzy sets through fuzzifiers. Fuzzy sets are usually used to process fuzzy or uncertain information. In the fuzzification process, it is necessary to first determine the membership function of the fuzzy set, and use the membership function to map the value of the element to the membership of the fuzzy set, indicating the degree to which the element belongs to the fuzzy set. The membership function needs to be symmetrical. The symmetry ensures that the fuzzifier's mapping of the input value and its neighboring values ​​is continuous and smooth. A triangular membership function is used to design each layer of fuzzy logic, as shown in the following formula:

[0102]

[0103] Among them, x represents the input signal of each layer of fuzzy logic, f(x) represents the output membership value, and a, b, and c are parameters.

[0104] In this application, the minimum, maximum, mean, variance, etc. of the input signal of each layer are calculated, and the membership function of each layer is specifically adjusted in the fuzzy control toolbox (Fuzzy Logic) of MATLAB using its data distribution.

[0105] 2) Fuzzy reasoning

[0106] Fuzzy reasoning is a process of reasoning and decision-making based on fuzzy sets, which is used to map input data to output data and provides an effective way to handle uncertainty and fuzzy knowledge. In this application, Mamdani reasoning method is selected to program imprecise, uncertain and unquantifiable concepts.

[0107] 3) Fuzzy rules

[0108] Fuzzy rules are a set of logical relationship expressions with causal reasoning. The premise and result are set artificially. The fuzzy value of the result is obtained by combining and reasoning multiple fuzzified input features, and then the required alert quantification value is obtained by defuzzification.

[0109] In the first layer of fuzzy logic, the corresponding fuzzy rule tables are shown in Tables 1 to 4:

[0110] Table 1 Fuzzy rules table of N200 signal in the first layer of fuzzy logic

[0111] Rule 1: If N200_A is LA or N200_T is BT, then N200 is NN.

[0112] Rule 2: If N200_A is LA or N200_T is AT, then N200 is LN.

[0113] Rule 3: If N200_A is LA or N200_T is AT, then N200 is LN.

[0114] Rule 4: If N200_A is MA or N200_T is BT, then N200 is NN.

[0115] Rule 5: If N200_A is MA or N200_T is OT, then N200 is MN.

[0116] Rule 6: If N200_A is MA or N200_T is AT, then N200 is LN.

[0117] Rule 7: If N200_A is HA or N200_T is BT, then N200 is MN.

[0118] Rule 8: If N200_A is HA or N200_T is OT, then N200 is HN.

[0119] Rule 9: If N200_A is HA or N200_T is AT, then N200 is MN.

[0120] Wherein, N200_A represents the amplitude of the N200 potential, and N200_T represents the latency of the N200 potential.

[0121] Table 2 Fuzzy rules table of P300 signal in the first layer of fuzzy logic

[0122] Rule 1: If P300_A is LA or P300_T is BT, then P300 is NP.

[0123] Rule 2: If P300_A is LA or P300_T is OT, then P300 is LP.

[0124] Rule 3: If P300_A is LA or P300_T is AT, then P300 is LP.

[0125] Rule 4: If P300_A is MA or P300_T is BT, then P300 is NP.

[0126] Rule 5: If P300_A is MA or P300_T is OT, then P300 is MP.

[0127] Rule 6: If P300_A is MA or P300_T is AT, then P300 is LP.

[0128] Rule 7: If P300_A is HA or P300_T is BT, then P300 is MP.

[0129] Rule 8: If P300_A is HA or P300_T is OT, then P300 is HP.

[0130] Rule 9: If P300_A is HA or P300_T is AT, then P300 is MP.

[0131] Wherein, P300_A represents the amplitude of P300 potential, and P300_T represents the latency of P300 potential.

[0132] Table 3 Fuzzy rules of event-related potential signals in the second-level fuzzy logic

[0133] Rule 1: If N100 is NN or P300 is NP, then ERP is NE.

[0134] Rule 2:If N100 is NN or P300 is LP,then ERP is LE.

[0135] Rule 3:If N100 is NN or P300 is MP,then ERP is ME.

[0136] Rule 4:If N100 is NN or P300 is HP,then ERP is HE.

[0137] Rule 5:If N100 is LN or P300 is NP,then ERP is LE.

[0138] Rule 6:If N100 is LN or P300 is LP,then ERP is LE.

[0139] Rule 7:If N100 is LN or P300 is MP,then ERP is ME.

[0140] Rule 8:If N100 is LN or P300 is HP,then ERP is HE.

[0141] Rule 9:If N100 is MN or P300 is NP,then ERP is ME.

[0142] Rule 10:If N100 is MN or P300 is LP,then ERP is ME.

[0143] Rule 11:If N100 is MN or P300 is MP,then ERP is ME.

[0144] Rule 12:If N100 is MN or P300 is HP,then ERP is HE.

[0145] Rule 13:If N100 is HN or P300 is NP,then ERP is HE.

[0146] Rule 14:If N100 is HN or P300 is LP,then ERP is HE.

[0147] Rule 15: If N100 is HN or P300 is MP, then ERP is HE.

[0148] Rule 16: If N100 is HN and P300 is HP, then ERP is HE.

[0149] Fuzzy rule table for vigilance quantification in the third layer of fuzzy logic in Table 4

[0150] Rule 1: If ERP is NE and PowerRatio is LR, then Vigilance is LV.

[0151] Rule 2: If ERP is NE and PowerRatio is MR, then Vigilance is LV.

[0152] Rule 3: If ERP is NE and PowerRatio is HR, then Vigilance is LV.

[0153] Rule 4: If ERP is LE and PowerRatio is LR, then Vigilance is LV.

[0154] Rule 5: If ERP is LE and PowerRatio is MR, then Vigilance is LV.

[0155] Rule 6: If ERP is LE and PowerRatio is HR, then Vigilance is MV.

[0156] Rule 7: If ERP is ME and PowerRatio is LR, then Vigilance is MV.

[0157] Rule 8: If ERP is ME and PowerRatio is MR, then Vigilance is MV.

[0158] Rule 9: If ERP is ME and PowerRatio is HR, then Vigilance is HV.

[0159] Rule 10: If ERP is HE and PowerRatio is LR, then Vigilance is HV.

[0160] Rule 11: If ERP is HE and PowerRatio is MR, then Vigilance is HV.

[0161] Rule 12: If ERP is HE and PowerRatio is HR, then Vigilance is HV.

[0162] Among them, Power Ratio represents the power ratio, and Vigilance represents the alertness.

[0163] 4) Defuzzification

[0164] Refers to the process of converting a fuzzy set into a specific numerical value after fuzzification and fuzzy rule operation in a fuzzy system, which is the description value of the degree of quantization required in this application. This application selects the maximum mean value method (MOM, middle of maximum) to obtain the defuzzified value.

[0165] This application proposes an alertness quantification model based on three layers of fuzzy logic: in the first layer of fuzzy logic, the input signal is the amplitude and latency of P300 and N200 extracted from the alertness EEG signal, and the output is 4 types of states (none, low, medium, and high); in the second layer of fuzzy logic, the input signal is the state of P300 and N200, and the output is 4 types of intensity values ​​of event-related potential characteristics (none, low, medium, and high); in the third layer of fuzzy logic, the input signal is the intensity value of the event-related potential and the power ratio extracted from the alertness EEG signal, and finally the output is 3 types of quantitative values ​​of alertness (high alertness, medium alertness, and fatigue). The schematic diagram of the alertness quantification model is shown below. Figure 7 shown.

[0166] In the first layer of fuzzy logic, the input signals are the amplitude and latency values ​​of P300 and N200 extracted from the alert EEG signals. Here, three types of amplitude states are defined, including low amplitude (LA), medium amplitude (MA), and high amplitude (HA), and three types of latency states, including early (BT), optimal time (OT), and delayed (AT). The output is the state values ​​of N200 and P300. P300 is divided into four states: no P300 (NP), low P300 (LP), medium P300 (MP), and high P300 (HP); N200 is divided into four states: no N200 (NN), low N200 (LN), medium N200 (MN), and high N200 (HN).

[0167] In the second layer of fuzzy logic, the state values ​​of P300 and N200 in the previous layer are input. Here, four types of event-related potential intensity values ​​are defined, namely, no event-related potential (NE), low event-related potential (LE), medium event-related potential (ME), and high event-related potential (HE). The OR operator is used to perform the operation as shown in formula (3), and the signal intensity value of the event-related potential is output.

[0168] ERP=(N100∨P300) (3).

[0169] Finally, in the third layer of fuzzy logic, the quantified value of alertness is determined based on the signal strength value of the event-related potential and the power ratio extracted from the alert EEG signal, which is defined as fatigue (LV), medium alertness (MV), and high alertness (HV). The event-related potential strength value and the power ratio PR are calculated as shown in formula (4):

[0170] V=ERP∧PR (4).

[0171] 5) Model parameter setting of alertness model based on three-layer fuzzy logic

[0172] The parameter settings of each layer of fuzzy logic of the alertness quantification model based on three-layer fuzzy logic are shown in the following table.

[0173] Table 5 Parameter settings of the first layer of fuzzy logic

[0174]

[0175] Table 6 Parameter settings of the second layer of fuzzy logic

[0176]

[0177] Table 7 Parameters of the third layer of fuzzy logic

[0178]

[0179]

[0180] 6) Alertness and fatigue labels based on behavioral performance

[0181] In this application, considering that the vigilance reduction effect is related to the task time, six 5-minute time windows are divided in the time domain. All subjects are averaged in each time window, and the obtained reaction time curve is as follows: Figure 8 One-way repeated measures ANOVA showed that there was a significant difference between the reaction time in the first time window and the last time window (F=20.9, p<0.001), and the difference increased over time.

[0182] Therefore, the objective division of alertness and fatigue states can be achieved in the time domain based on the difference between the first time window and the last time window, and subsequent analysis will achieve the quantification of alertness every 5 minutes.

[0183] VII) Alertness Quantification Results

[0184] The alertness quantification results in the first and sixth time windows of the four MCT experiments for each subject are shown in Table 8. The quantification value intervals [0, 4], [4, 8], and [8, 10] are selected as the threshold ranges of fatigue, medium alertness, and high alertness, respectively. From the results, it can be seen that the alertness quantification model based on three-layer fuzzy logic proposed in this application can clarify the mapping relationship of alertness quantification and realize the three classifications of high alertness, medium alertness, and fatigue.

[0185] According to the behavioral manifestation of the vigilance reduction effect, this application selects the change between the first and last time windows as the negative value, and the obtained vigilance change and discrimination accuracy of each subject are shown in Table 9. The average accuracy of the model reached 90.9%. Among them, the accuracy of subject 8 was only 50%, which may be due to extreme sleepiness during the experiment, and the correct rate of keystrokes in the later stage of each vigilance task was extremely low.

[0186] Table 8 Quantified values ​​of alertness in the first and sixth time windows of the four experiments for each subject

[0187]

[0188] Table 9 Changes in alertness and accuracy of each subject in four experiments

[0189]

[0190] The vigilance quantification method based on three-layer fuzzy logic proposed in this application determines a clear mapping relationship of vigilance quantification by combining and reasoning multiple electrophysiological feature inputs related to the vigilance reduction effect, and uses the objective reaction time behavior induced by long-term tasks as labels to achieve three classifications of high vigilance, medium vigilance, and fatigue. The average accuracy of the model reaches 90.9%. This method clarifies the mapping relationship of vigilance quantification and has strong interpretability.

[0191] Fig. 9 is a schematic diagram of the structure of the alertness quantification device provided in the embodiment of the present application, such as Fig. 9As shown, the alertness quantification device 900 includes: an acquisition module 901, which is used to obtain the target EEG signal data of the person to be monitored; an extraction module 902, which is used to extract features of the target EEG signal data to obtain event-related potential features and power ratio features; the event-related potential features at least include P300 features and N200 features; a judgment module 903, which is used to make state judgments on the amplitudes and latency of the P300 features and the N200 features based on a pre-constructed alertness quantification model to obtain state values; the judgment module 903 is also used to make intensity judgments on the state values ​​based on the alertness quantification model to obtain signal strength values; a quantification module 904, which is used to perform alertness quantification on the signal strength values ​​and the power ratio features based on the alertness quantification model to obtain alertness quantification results of the person to be monitored.

[0192] In some embodiments, the acquisition module 901 is also used to obtain the initial EEG signal data of the person to be tested; the initial EEG signal data is processed in sequence by average reference, low-frequency filtering, artifact removal, data segmentation and baseline correction to obtain the target EEG signal data.

[0193] In some embodiments, the extraction module 902 is further used to divide the target EEG signal data into multiple frequency bands; and use a preset power spectrum density estimation method to calculate the multiple frequency bands to obtain the power ratio feature.

[0194] In some embodiments, the alertness quantification model includes a first layer of fuzzy logic; the judgment module 903 is also used to input the amplitude and latency of the P300 feature and the N200 feature into the first layer of fuzzy logic for state judgment to obtain the state value.

[0195] In some embodiments, the alertness quantification model includes a second layer of fuzzy logic; the judgment module 903 is further used to input the state value into the second layer of fuzzy logic to perform strength judgment to obtain the signal strength value.

[0196] In some embodiments, the alertness quantification model includes a third layer of fuzzy logic; the judgment module 903 is further used to input the state value into the third layer of fuzzy logic to perform alertness quantification to obtain the alertness quantification result.

[0197] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0198] It should be noted that in the embodiments of the present application, if the above-mentioned alertness quantification method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application can be essentially or partly reflected in the form of a software product, which is stored in a storage medium and includes several instructions for a terminal to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk. In this way, the embodiments of the present application are not limited to any specific combination of hardware and software.

[0199] Correspondingly, the embodiment of the present application provides a device for quantifying alertness. Fig.10 is a schematic diagram of the composition structure of the alertness quantification device provided in an embodiment of the present application, such as Fig.10 As shown, the alertness quantification device 1000 at least includes: a processor 1001 and a computer-readable storage medium 1002 configured to store executable instructions, wherein the processor 1001 generally controls the overall operation of the alertness quantification device 1300. The computer-readable storage medium 1002 is configured to store instructions and applications executable by the processor 1001, and can also cache data to be processed or processed by the processor 1001 and each module in the alertness quantification device 1000, which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM).

[0200] The present application embodiment provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the present application embodiment, for example, Figure 2 The method shown.

[0201] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above memories.

[0202] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0203] As an example, executable instructions may, but need not necessarily, correspond to a file in a file system, may be stored as part of a file storing other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0204] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

[0205] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0206] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of further restrictions, an element defined by the statement "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed.

[0207] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for quantifying alertness, characterized in that: The method comprises: Obtaining target EEG signal data of the person to be monitored; Extracting features of the target EEG signal data to obtain event-related potential features and power ratio features; the event-related potential features at least include P300 features and N200 features; Based on a pre-built alertness quantification model, the amplitude and latency of the P300 feature and the N200 feature are judged to obtain a state value; Based on the alertness quantification model, performing strength judgment on the state value to obtain a signal strength value; Based on the alertness quantification model, the signal strength value and the power ratio feature are quantified to obtain an alertness quantification result of the person to be monitored.

2. The method according to claim 1, characterized in that The step of obtaining the EEG signal data of the person to be monitored includes: Acquiring initial EEG signal data of the person to be detected; The initial EEG signal data is processed in sequence by average reference, low frequency filtering, artifact removal, data segmentation and baseline correction to obtain the target EEG signal data.

3. The method according to claim 1, characterized in that: Performing feature extraction on the target EEG signal data to obtain power ratio features includes: Dividing the target EEG signal data into multiple frequency bands; A preset power spectrum density estimation method is used to calculate the multiple frequency bands to obtain the power ratio feature.

4. The method according to claim 1, characterized in that: The alertness quantification model includes a first layer of fuzzy logic; The state judgment of the amplitude and latency of the P300 feature and the N200 feature based on the pre-built alertness quantification model to obtain the state value includes: The amplitudes and latency of the P300 feature and the N200 feature are respectively input into the first layer of fuzzy logic for state judgment to obtain the state value.

5. The method according to claim 1, characterized in that The alertness quantification model includes a second layer of fuzzy logic; The step of performing strength judgment on the state value based on the alertness quantification model to obtain a signal strength value includes: The state value is input into the second layer fuzzy logic to perform strength judgment to obtain the signal strength value.

6. The method according to claim 1, characterized in that The alertness quantification model includes a third layer of fuzzy logic; The step of performing alertness quantification on the signal strength value and the power ratio feature based on the alertness quantification model to obtain an alertness quantification result of the person to be monitored includes: The state value is input into the third layer fuzzy logic to perform alertness quantification to obtain the alertness quantification result.

7. A device for quantifying alertness, characterized in that: The device comprises: An acquisition module, used to obtain target EEG signal data of the person to be monitored; An extraction module, used for performing feature extraction on the target EEG signal data to obtain event-related potential features and power ratio features; the event-related potential features at least include P300 features and N200 features; A judgment module, used to make a state judgment on the amplitude and latency of the P300 feature and the N200 feature based on a pre-built alertness quantification model to obtain a state value; The judgment module is further used to perform strength judgment on the state value based on the alertness quantification model to obtain a signal strength value; The quantification module is used to perform alertness quantification on the signal strength value and the power ratio feature based on the alertness quantification model to obtain the alertness quantification result of the person to be monitored.

8. A device for quantifying alertness, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the alertness quantification method according to any one of claims 1 to 6 when executing executable instructions stored in the memory.

9. A computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the alertness quantification method according to any one of claims 1 to 6.

Images