Driver mental monitoring method and system based on electroencephalogram signals

Abstract

The application discloses a driver psychological monitoring method and system based on electroencephalogram signals, real-time acquisition of the driver's head electroencephalogram signals through a multi-channel electroencephalogram device, extraction of multi-dimensional features after pretreatment, and accurate identification of emotional states such as tension, fatigue and anger by using a support vector machine model. The system adopts a dynamic threshold determination mechanism, combines sliding time window statistical analysis and composite rule verification, and adaptively adjusts the emotional abnormality standard, thereby significantly reducing the influence of environmental interference. Based on the real-time monitoring result, a multi-level safety intervention strategy is triggered, and the response is gradually performed from voice warning, speed limitation to vehicle forced takeover, so that the driving autonomy and safety are balanced. The application has high precision, strong robustness and scene adaptability, can effectively reduce the traffic accident risk caused by abnormal driver psychology, and provides an innovative solution for the development of intelligent traffic management and active safety technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electroencephalogram data processing, in particular to a driver psychological monitoring method and system based on electroencephalogram signals. BACKGROUND

[0002] In today's rapidly developing society, road traffic safety problems are like a sword hanging over the process of social development, threatening human life and property safety at any time. With the vigorous development of the automobile industry and the rapid increase in the number of private cars, the road traffic network is becoming increasingly complex, and the traffic flow is surging, leading to an increase in the incidence of traffic accidents. According to the statistics of the World Health Organization and many national traffic departments, the number of deaths caused by road traffic accidents each year is staggering, and this figure continues to be high. In-depth analysis shows that although road facilities are constantly improving and vehicle safety technology is increasingly improving, the driver as the core component of the traffic system, the fluctuations and changes in his psychological state are still one of the important reasons for the frequent occurrence of traffic accidents.

[0003] Electroencephalogram signals can capture the neural activity patterns of the brain in processing external information, generating emotional responses, and cognitive processing by recording the weak potential changes on the scalp surface. These patterns not only reveal the real-time state of brain function, but also reflect the psychological response and behavioral tendency of the driver in a specific situation. Therefore, driver psychological monitoring technology based on electroencephalogram signals is expected to provide objective and quantitative indicators for assessing the psychological state of the driver and provide scientific basis for realizing personalized and precise driving safety intervention. SUMMARY

[0004] In order to solve the above technical problems, the technical scheme adopted by the driver psychological monitoring method based on electroencephalogram signals includes the following steps:

[0005] S1: Electroencephalogram signal acquisition and preprocessing, real-time acquisition of raw electroencephalogram signals through a multi-channel electroencephalogram acquisition device deployed on the driver's head, the device contains at least 8 electrodes covering the frontal lobe, parietal lobe and temporal lobe regions, and the signal acquisition is carried out at a sampling frequency of not less than 256Hz, and the raw signal is subjected to preprocessing operation;

[0006] S2: Electroencephalogram signal feature extraction, multi-dimensional feature extraction is performed on the preprocessed electroencephalogram signals, specifically including:

[0007] (a) Calculate the power spectral density of each channel based on the fast Fourier transform, and extract the energy values of the delta wave, theta wave, alpha wave, beta wave and gamma wave frequency bands respectively, and calculate the percentage of the energy of each frequency band to the total frequency band energy;

[0008] (b) wavelet decomposition of the signal, extraction of the time-frequency energy distribution of the theta wave 3rd layer detail coefficient and the beta wave 4th layer detail coefficient, and calculation of the variance thereof as a time-frequency domain feature;

[0009] (c) calculating the frontal lobe region channel and channel wave energy value, parietal lobe region channel and channel wave energy value, and calculating the electroencephalogram asymmetry index:

[0010] , wherein and are the frontal lobe region channel and channel wave energy value, and are the parietal lobe region channel and channel wave energy value;

[0011] S3: mental state classification, the δ wave energy proportion, the θ wave energy proportion, the β wave energy proportion, the γ wave energy proportion, the θ wave time-frequency energy variance, the β wave time-frequency energy variance and the FAA index constitute a feature vector, which is input into a pre-trained mental state classification model, and the current mental state category of the driver is output, wherein:

[0012] (1) the classification model adopts a support vector machine (SVM) architecture, and the training data set contains electroencephalogram samples labeled as four emotions of tension, fatigue, anger and calmness;

[0013] (2) the classification model outputs a probability distribution of the four emotions, and when the probability value of a certain emotion category is ≥0.75 and the classification results of three consecutive time windows are consistent for 6 seconds, it is determined that the emotion state is valid;

[0014] (3) when the anger or fatigue state is valid, a dynamic emotion threshold determination process is triggered;

[0015] S4: dynamic emotion threshold determination, dynamically adjusting the emotion abnormality threshold according to the real-time classification result and the historical data, specifically including:

[0016] S41: establishing a sliding time window with a length of 60 seconds, storing the intensity index (such as the β wave energy proportion corresponding to anger) of the current valid emotion, and calculating the mean μ and the standard deviation σ of the data in the window;

[0017] S42: defining the emotion abnormality condition as the real-time emotion intensity index exceeding μ+2σ, and setting a composite determination rule according to different emotion types:

[0018] Tension state: θ wave energy ratio ≥ 35% and FAA index ≤-0.2 for 10 seconds;

[0019] Anger state: β wave energy ratio ≥ 40% and the growth rate of γ wave energy in the temporal lobe region within 1 second is ≥ 20%;

[0020] Fatigue state: α wave energy ratio ≥ 50% and the eyelid closure frequency detected by the vehicle-mounted camera is ≥ 0.3Hz;

[0021] S43: when the composite determination rule is met, a preset multi-level intervention strategy is matched according to the emotion category;

[0022] S5: multi-level intervention strategy execution, triggering hierarchical control instructions according to the emotion category and the abnormal level, including:

[0023] First-level intervention: when the α wave energy ratio is 50%-60%, it is detected as tension or mild fatigue, and mild perceptual intervention is performed, and a voice prompt is sent to the driver;

[0024] Second-level intervention: when the α wave energy ratio is 60%-70%, it is detected as anger or moderate fatigue, and the environmental restriction is increased on the basis of the first-level intervention, and at the same time, an instruction is sent to the engine control unit ECU through the vehicle CAN bus, limiting the vehicle speed to 80% of the current road speed limit value;

[0025] Third-level intervention: when the β wave energy ratio is ≥ 40% for more than 30 seconds, it is detected as continuous anger or severe fatigue, and when the α wave energy ratio is ≥ 70%, the driver's control authority on the accelerator pedal is locked, the brake function is reserved, the automatic driving system is started to take over the vehicle control, and an alarm signal containing the driver ID, geographical position and emotion data is sent to the cloud monitoring platform through the vehicle-mounted communication module.

[0026] As a further scheme of the present application, the preprocessing operation in S1 includes:

[0027] (a) using a 0.5Hz to 60Hz band-pass filter to eliminate low-frequency motion artifacts and high-frequency noise, and superimposing a 50Hz or 60Hz notch filter to suppress power frequency interference;

[0028] (b) separating the electrooculogram artifact and electromyogram artifact parts in the signal, and reconstructing the electroencephalogram signal; the reconstructed signal is divided into data segments of 2 seconds in length according to a fixed time window, and the baseline correction of each data segment is subtracted from the mean value of the segment and normalized to the range of [-1, 1].

[0029] As a further scheme of the present application, in S2, the power spectrum band of each channel is: delta wave: 0.5 Hz to 4 Hz, theta wave: 4 Hz to 8 Hz, alpha wave: 8 Hz to 13 Hz, beta wave: 13 Hz to 30 Hz, and gamma wave: 30 Hz to 60 Hz;

[0030] As a further scheme of the present application, the driver psychological monitoring system based on the electroencephalogram signal, the system module comprises:

[0031] The electroencephalogram signal acquisition module is responsible for collecting the original electroencephalogram signal through the multi-channel electroencephalogram acquisition device deployed on the head of the driver, and performing baseline correction and normalization preprocessing on the data;

[0032] The feature extraction module: multi-dimensional feature extraction is performed on the preprocessed electroencephalogram signal, including calculating the power spectrum density of each channel, extracting the energy value and proportion of different frequency bands, performing wavelet decomposition to extract the time-frequency energy distribution and calculate the variance, and calculating the electroencephalogram asymmetry index;

[0033] The psychological state classification module: the extracted feature vector is input into the pre-trained psychological state classification model using the support vector machine SVM architecture, and the probability distribution of the current psychological state category of the driver is output, and the emotion state is determined according to the set rule whether it is effective;

[0034] The dynamic emotion threshold determination module: according to the real-time classification result and the historical data, the mean and standard deviation of the data in the sliding time window are calculated by establishing a sliding time window, the emotion abnormal condition and the composite determination rule are defined, and it is judged whether the emotion abnormal condition is met;

[0035] The multi-level intervention strategy execution module: according to the emotion category and the abnormal level, the hierarchical control instruction is triggered, and the intervention operation of different levels is executed.

[0036] Beneficial effects: The present application effectively improves driving safety and the accuracy of psychological state monitoring by real-time acquisition and analysis of driver brain electrical signals, combined with dynamic threshold determination and multi-level intervention strategies. The system can capture subtle changes in brain neural activity, accurately identify emotional states such as tension, fatigue, anger, etc. using multi-dimensional feature fusion technology, and reduce environmental noise interference through an adaptive dynamic threshold mechanism, significantly reducing the misjudgment rate. Based on real-time monitoring results, the system intelligently triggers graded intervention measures, from voice reminders, speed limits to vehicle forced takeover, gradually strengthening safety protection, retaining driver autonomy while actively avoiding risks in extreme emotions. At the same time, by integrating vehicle-mounted cameras, vehicle control buses and cloud communication modules, the system realizes multi-modal data collaborative analysis and remote monitoring, providing comprehensive protection for driving behavior. The system has good environmental adaptability and expansion capability, can be flexibly applied to different driving scenarios, provides important support for intelligent traffic management and active safety technology development, and has significant social value in reducing traffic accident rates and improving road safety levels. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The step flow chart of the driver psychological monitoring method based on brain electrical signals of the present application is shown in the figure.

[0038] Figure 2 The module schematic diagram of the driver psychological monitoring system based on brain electrical signals of the present application is shown in the figure. DETAILED DESCRIPTION

[0039] The present application will be further described in detail below in conjunction with the embodiments.

[0040] Please refer to Figure 1 shown in the figure, Figure 1 The step flow chart of the driver psychological monitoring method based on brain electrical signals of the present application is shown in the figure, and the specific steps include:

[0041] S1: Brain electrical signal acquisition and preprocessing, real-time acquisition of original brain electrical signals through multi-channel brain electrical acquisition equipment deployed on the driver's head, the equipment contains at least 8 electrodes covering the frontal lobe, parietal lobe and temporal lobe regions, and the signal acquisition is carried out at a sampling frequency not less than 256Hz, and the original signal is subjected to preprocessing operation;

[0042] S2: Brain electrical signal feature extraction, multi-dimensional feature extraction is performed on the preprocessed brain electrical signal, specifically including:

[0043] (a) Calculate the power spectral density of each channel based on fast Fourier transform, respectively extract the energy values of delta wave, theta wave, alpha wave, beta wave and gamma wave frequency bands, and calculate the percentage of each frequency band energy in total frequency band energy;

[0044] (b) wavelet decomposition of the signal, extraction of the time-frequency energy distribution of the theta wave 3rd layer detail coefficient and the beta wave 4th layer detail coefficient, and calculation of the variance thereof as a time-frequency domain feature;

[0045] (c) calculating the frontal lobe region channel and channel wave energy value, parietal lobe region channel and channel wave energy value, and calculating the electroencephalogram signal asymmetry index:

[0046] , wherein and are the frontal lobe region channel and channel wave energy value, and are the parietal lobe region channel and channel wave energy value;

[0047] S3: mental state classification, constructing a feature vector with the delta wave energy proportion, theta wave energy proportion, beta wave energy proportion, gamma wave energy proportion, theta wave time-frequency energy variance, beta wave time-frequency energy variance and FAA index, inputting the pre-trained mental state classification model, and outputting the current mental state category of the driver, wherein:

[0048] (1) the classification model adopts a support vector machine (SVM) architecture, and the training data set contains electroencephalogram samples labeled as four emotions of tension, fatigue, anger and calmness;

[0049] (2) the classification model outputs a probability distribution of the four emotions, and when the probability value of a certain emotion category is ≥0.75 and the classification results of three consecutive time windows are consistent for 6 seconds, it is determined that the emotion state is valid;

[0050] (3) when the anger or fatigue state is valid, a dynamic emotion threshold determination process is triggered;

[0051] S4: dynamic emotion threshold determination, dynamically adjusting the emotion abnormality threshold according to the real-time classification results and historical data, specifically including:

[0052] S41: establishing a sliding time window with a length of 60 seconds, storing the intensity index (such as the beta wave energy proportion corresponding to anger) of the current valid emotion, and calculating the mean μ and standard deviation σ of the data in the window;

[0053] S42: defining the emotion abnormality condition as the real-time emotion intensity index exceeding μ+2σ, and setting a composite determination rule according to different emotion types:

[0054] Tension state: θ wave energy ratio ≥ 35% and FAA index ≤-0.2 for 10 seconds;

[0055] Anger state: β wave energy ratio ≥ 40% and the growth rate of γ wave energy in the temporal lobe region within 1 second is ≥ 20%;

[0056] Fatigue state: α wave energy ratio ≥ 50% and eyelid closure frequency detected by the vehicle-mounted camera is ≥ 0.3 Hz;

[0057] S43: When the composite decision rule is met, match the preset multi-level intervention strategy according to the emotion category;

[0058] S5: Multi-level intervention strategy execution, trigger hierarchical control instructions according to emotion category and abnormal level, including:

[0059] First-level intervention: α wave energy ratio 50%-60%, detected as tension or mild fatigue, mild perceptual intervention is performed, and a voice prompt is sent to the driver;

[0060] Second-level intervention: α wave energy ratio 60%-70%, detected as anger or moderate fatigue, environmental constraints are added on the basis of first-level intervention, and instructions are sent to the engine control unit ECU through the vehicle CAN bus to limit the vehicle speed to 80% of the current road speed limit;

[0061] Third-level intervention: β wave energy ratio ≥ 40% for more than 30 seconds, detected as persistent anger or severe fatigue, and when α wave energy ratio ≥ 70%, lock the driver's control authority over the accelerator pedal, retain the brake function, start the automatic driving system to forcibly take over vehicle control, and send an alarm signal containing the driver ID, geographic location and emotional data to the cloud monitoring platform through the vehicle-mounted communication module.

[0062] Further, the preprocessing operation in S1 includes:

[0063] (a) Use a 0.5 Hz to 60 Hz band-pass filter to eliminate low-frequency motion artifacts and high-frequency noise, and superimpose a 50 Hz or 60 Hz notch filter to suppress power frequency interference;

[0064] (b) Separate the electrooculogram artifact and electromyogram artifact parts in the signal, and reconstruct the electroencephalogram signal; the reconstructed signal is divided into 2-second length data segments according to a fixed time window, and baseline correction is performed on each data segment to subtract the mean value of the segment and normalized to the range of [-1, 1].

[0065] Further, in S2, the power spectrum frequency bands of each channel are as follows: delta wave: 0.5 Hz to 4 Hz, theta wave: 4 Hz to 8 Hz, alpha wave: 8 Hz to 13 Hz, beta wave: 13 Hz to 30 Hz, and gamma wave: 30 Hz to 60 Hz;

[0066] Further, please refer to Figure 2 as shown, Figure 2 is a schematic diagram of a driver psychological monitoring system module based on the brain electrical signal of the application, and the system module comprises:

[0067] The brain electrical signal acquisition module is responsible for collecting raw brain electrical signals through a multi-channel brain electrical signal acquisition device deployed on the head of the driver, and performing baseline correction and normalization preprocessing on the data.

[0068] The feature extraction module performs multi-dimensional feature extraction on the preprocessed brain electrical signals, including calculating the power spectrum density of each channel, extracting the energy values and proportions of different frequency bands, performing wavelet decomposition to extract time-frequency energy distribution and calculate variance, and calculating the brain electrical asymmetry index.

[0069] The psychological state classification module inputs the extracted feature vector into a pre-trained psychological state classification model using a support vector machine (SVM) architecture, outputs the probability distribution of the current psychological state category of the driver, and determines whether the emotional state is effective according to the set rules.

[0070] The dynamic emotional threshold determination module calculates the mean and standard deviation of the data in the window by establishing a sliding time window according to the real-time classification results and historical data, defines emotional abnormality conditions and composite determination rules, and determines whether the emotional abnormality conditions are met.

[0071] The multi-level intervention strategy execution module triggers a hierarchical control instruction according to the emotional category and abnormality level, and executes different levels of intervention operations.

[0072] The above is only an embodiment of the present application and is not intended to limit the present application. Those skilled in the art can make various changes and modifications to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of the claims of the present application.

Claims

1. A method for monitoring the mental state of a driver based on electroencephalogram signals, characterized in that, The method comprises the following steps: S1: Electroencephalogram signal acquisition and preprocessing, real-time acquisition of raw electroencephalogram signals through a multi-channel electroencephalogram acquisition device deployed on the head of the driver, the device comprising at least 8 electrodes covering the frontal lobe, parietal lobe and temporal lobe regions, and performing signal acquisition at a sampling frequency of not less than 256 Hz, and performing preprocessing operations on the raw signals; S2: Electroencephalogram signal feature extraction, multi-dimensional feature extraction is performed on the preprocessed electroencephalogram signals, specifically including: (a) calculating the power spectral density of each channel based on fast Fourier transform, respectively extracting the energy values of delta wave, theta wave, alpha wave, beta wave and gamma wave frequency bands, and calculating the percentage of energy of each frequency band to the total frequency band energy; (b) wavelet decomposition is performed on the signal, the time-frequency energy distribution of the third layer detail coefficient of theta wave and the fourth layer detail coefficient of beta wave is extracted, and the variance thereof is calculated as the time-frequency domain feature; (c) according to the frontal region channel and channel wave energy values, parietal region channel and channel of wave energy values, calculating an asymmetry index of the electroencephalographic signal: , wherein and is a frontal region channel and channel of wave energy values, and is a parietal region channel and channel of wave energy values; S3: Psychological state classification, the energy percentage of delta wave, the energy percentage of theta wave, the energy percentage of beta wave, the energy percentage of gamma wave, the time-frequency energy variance of theta wave, the time-frequency energy variance of beta wave and the FAA index constitute a feature vector, which is input into a pre-trained psychological state classification model, and the current psychological state category of the driver is output, wherein: (1) the classification model adopts a support vector machine (SVM) architecture, and the training data set comprises electroencephalogram samples labeled as four types of emotions: nervousness, fatigue, anger and calmness; (2) the classification model outputs the probability distribution of the four types of emotions, when the probability value of a certain emotion category is greater than or equal to 0.75 and the classification results of three consecutive time windows are consistent for 6 seconds, it is determined that the emotion state is valid; (3) when the anger or fatigue state is valid, a dynamic emotion threshold determination process is triggered; S4: Dynamic emotion threshold determination, dynamically adjusting the emotion abnormal threshold according to the real-time classification results and historical data, specifically including: S41: establishing a sliding time window with a length of 60 seconds, storing the intensity index of the current valid emotion, calculating the mean μ and standard deviation σ of the data in the window; S42: defining the emotion abnormal condition as the real-time emotion intensity index exceeding μ+2σ, and setting a composite judgment rule according to different emotion types: Nervous state: θ wave energy percentage ≥ 35% and FAA index ≤-0.2 for 10 seconds; Anger state: β wave energy percentage ≥ 40% and γ wave energy in the temporal lobe region increases by ≥ 20% within 1 second; Fatigue state: α wave energy percentage ≥ 50% and eyelid closure frequency detected by the vehicle-mounted camera is ≥ 0.3 Hz; S43: when the composite judgment rule is satisfied, a multi-level intervention strategy is matched according to the emotion category; S5: Multi-level intervention strategy execution, triggering hierarchical control instructions according to the emotion category and abnormal level, including: First-level intervention: when the α wave energy percentage is 50%-60%, the driver is detected to be nervous or slightly fatigued, and a light perception intervention is performed, and a voice prompt is sent to the driver; Second-level intervention: when the α wave energy percentage is 60%-70%, the driver is detected to be angry or moderately fatigued, and environmental constraints are added on the basis of the first-level intervention, and instructions are sent to the engine control unit (ECU) through the vehicle CAN bus to limit the vehicle speed to 80% of the current road speed limit; Three-level intervention: When the proportion of beta wave energy is greater than or equal to 40% for more than 30 seconds, the driver is detected to be in a state of continuous anger or severe fatigue, and when the proportion of alpha wave energy is greater than or equal to 70%, the driver's control authority over the accelerator pedal is locked, the brake function is retained, the automatic driving system is started to forcibly take over the control of the vehicle, and an alarm signal containing the driver ID, geographical location and emotional data is sent to the cloud monitoring platform through the vehicle communication module.

2. The brain electrical signal based driver mental monitoring method according to claim 1, characterized in that, The preprocessing operation in S1 includes: (a) using a 0.5 Hz to 60 Hz band-pass filter to eliminate low-frequency motion artifacts and high-frequency noise, and superimposing a 50 Hz or 60 Hz notch filter to suppress power frequency interference; (b) separating the electrooculogram artifact and electromyogram artifact parts in the signal, and reconstructing the electroencephalogram signal; the reconstructed signal is divided into data segments of 2 seconds in length according to a fixed time window, and the baseline correction of each data segment is performed to subtract the mean value of the segment and normalized to the range of [-1, 1].

3. The brain electrical signal based driver mental monitoring method according to claim 1, characterized in that, In S2, the power spectrum frequency band of each channel is: delta wave: 0.5 Hz to 4 Hz, theta wave: 4 Hz to 8 Hz, alpha wave: 8 Hz to 13 Hz, beta wave: 13 Hz to 30 Hz, and gamma wave: 30 Hz to 60 Hz.

4. A driver mental monitoring system based on electroencephalogram signals, performing the driver mental monitoring method based on electroencephalogram signals according to any one of claims 1-3, characterized in that, The system comprises: An electroencephalogram signal acquisition module: responsible for acquiring raw electroencephalogram signals through a multi-channel electroencephalogram acquisition device deployed on the driver's head, and performing baseline correction and normalization preprocessing on the data; A feature extraction module: performing multi-dimensional feature extraction on the preprocessed electroencephalogram signals, including calculating the power spectral density of each channel, extracting the energy values and proportions of different frequency bands, performing wavelet decomposition to extract time-frequency energy distribution and calculate variance, and calculating the electroencephalogram asymmetry index; A psychological state classification module: inputting the extracted feature vector into a pre-trained psychological state classification model using a support vector machine (SVM) architecture, outputting the probability distribution of the current psychological state category of the driver, and determining whether the emotional state is effective according to the set rules; A dynamic emotional threshold determination module: according to the real-time classification results and historical data, a sliding time window is established, the mean and standard deviation of the data in the window are calculated, the emotional abnormality condition and compound determination rule are defined, and it is determined whether the emotional abnormality condition is met; A multi-level intervention strategy execution module: triggering a hierarchical control instruction according to the emotional category and abnormality level, and executing different levels of intervention operations.

Images