Driver drowsiness detection method, electronic equipment and storage medium

Abstract

The invention discloses a driver drowsiness detection method, electronic equipment and a storage medium. The electronic equipment collects electroencephalogram data of a driver in a driving state; performing drowsiness detection on the electroencephalogram data by using the trained drowsiness detection model to obtain a drowsiness detection result of the driver; the drowsiness detection result represents a drowsiness degree result of the driver in the current driving state; the trained drowsiness detection model is obtained by training the initial drowsiness detection model by using a training data set obtained based on a preset experiment task; the preset experiment tasks comprise a first preset task executed under static stimulation, a second preset task executed under dynamic stimulation and a third preset task used for determining a drowsiness level; the first preset task comprises a Carloin ska drowsiness test task and an alpha attenuation test task; the second preset task comprises a visual P300 test task and an auditory P300 test task; the third preset task comprises a mental movement warning task and a subjective scale task.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a driver drowsiness detection method, electronic equipment and storage medium. Background Art

[0002] The driver's drowsiness directly affects his driving safety. Among the relevant drowsiness detection methods, most are non-contact detection, that is, the detection method does not require the driver to wear any sensors that touch the body. The advantage of this method is that it is easy to use and does not require any additional operation from the driver. However, this method is easily affected by various factors and has many inconveniences when used. For example, the camera for detecting drowsiness is greatly affected by light, or some drivers can enter a micro-sleep state with their eyelids open normally, and the driver cannot wear sunglasses, masks or other obstructions during the detection process. In this case, contact detection shows obvious advantages, that is, the driver needs to wear sensors that touch the body to collect physiological signals.

[0003] However, most of the current contact detection systems are based on the driver's eyes and body posture, which are easily affected by the environment and individual differences, which will affect the detection accuracy and cannot be applied to all drivers. In addition, a threshold is often used to determine whether the driver is fatigued, but the drowsiness state may manifest differently for different drivers. Therefore, the threshold judgment method may not accurately reflect the driver's drowsiness, resulting in a decrease in detection accuracy. In other words, how to improve the detection accuracy of the driver's drowsiness is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide a driver drowsiness detection method, an electronic device, and a storage medium, which can effectively improve the detection accuracy of the driver's drowsiness.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for detecting driver drowsiness, the method comprising:

[0007] Collect the driver's EEG data while driving;

[0008] Performing drowsiness detection on the EEG data using the trained drowsiness detection model to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task;

[0009] The preset experimental tasks include a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the sleepiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

[0010] In the above embodiment, a training data set is obtained by using a first preset task performed under static stimulation, a second preset task performed under dynamic stimulation, and a third preset task for determining the sleepiness level, and the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task; these tasks can fully simulate various types of stimulations to which the driver is subjected in a real driving scenario, so that In order to obtain relevant data reflecting the driver's drowsiness in real driving scenarios, the initial drowsiness detection model is trained based on the training data set composed of various types of data that are close to the possible drowsiness in real driving scenarios, which can effectively improve the detection accuracy of the driver's drowsiness by the trained drowsiness detection model; thus, after the trained drowsiness detection model performs drowsiness detection on the driver's EEG data, the obtained drowsiness detection result can characterize the driver's drowsiness level in the current driving state, that is, the drowsiness detection result can accurately reflect the driver's current drowsiness level, thereby achieving accurate detection of the driver's drowsiness.

[0011] Furthermore, in some embodiments of the present application, the method further includes:

[0012] Collecting EEG data and eye movement data of the experimental target under the first preset task, the second preset task, and the third preset task to obtain a first data set;

[0013] According to the experimental objective, the subjective scale data under the Karolinska Sleepiness Scale task and the Visual Analog Scale task are calibrated for sleepiness grading on the electroencephalogram data under the Karolinska Sleepiness Test task, the Alpha Decay Test task, the Visual P300 test, and the Auditory P300 test task in the first data set to obtain calibrated electroencephalogram data;

[0014] Performing data cleaning processing on the calibrated EEG data based on the eye movement data in the first data set to obtain processed EEG data;

[0015] Feature extraction is performed based on the processed EEG data to obtain the training data set.

[0016] In the above embodiment, when constructing a training data set, first, the EEG data and eye movement data of the experimental target during the first preset task, the second preset task and the third preset task are collected to obtain a first data set, and then the subjective scale data obtained when the experimental target performs the Karolinska Sleepiness Scale task and the Visual Analog Scale task are used to calibrate the sleepiness grade of the EEG data under the Karolinska Sleepiness Test Task, the Alpha Decay Test Task, the Visual P300 Test Task and the Auditory P300 Test Task in the first data set to obtain the calibrated EEG data, and then the eye movement data in the first data set are used to perform data cleaning processing on the calibrated EEG data to obtain the processed EEG data, and finally feature extraction processing is performed based on the processed EEG data to obtain the training data set. It can be seen that the construction of the training data set is calibrated by the subjective scale data and cleaned based on the objective behavior data, such as the eye movement data, that is, the EEG data in the training data set is obtained by combining the subjective and objective methods, which can effectively improve the accuracy of the labels corresponding to the calibrated EEG data in the training data set.

[0017] Furthermore, in some embodiments of the present application, the step of performing data cleaning on the calibrated EEG data based on the eye movement data in the first data set includes:

[0018] Determining target eye movement data according to the eye movement data in the first data set; wherein the target eye movement data includes eye movement data under the psychomotor vigilance task, the subjective scale task, the Karolinska sleepiness test task in the first state, and the alpha decay test task; the first state represents that the experimental target is in an open-eye state;

[0019] Performing sleepiness grading calibration on the target eye movement data according to the subjective scale data to obtain calibrated eye movement data;

[0020] Calculating a first average value corresponding to the eye movement data at each sleepiness level in the calibrated eye movement data; wherein the first average value includes an average value calculated based on the eyelid closure degree data in the eye movement data;

[0021] The calibrated EEG data is cleaned based on the first average value.

[0022] In the above embodiment, when using eye movement data to perform data cleaning processing on EEG data, the required target eye movement data is first selected from the eye movement data, and the target eye movement data includes eye movement data under psychomotor vigilance task, subjective scale task, Karolinska sleepiness test task and alpha decay test task in which the experimental target is in an open-eyed state; then, the subjective scale data is used to calibrate the target eye movement data for sleepiness classification, and a first average value is calculated based on the calibrated eye movement data, so as to use the first average value to perform data cleaning processing on the calibrated EEG data, so as to eliminate data in the calibrated EEG data that differs greatly from the first average value, thereby improving the reliability of the processed EEG data.

[0023] Furthermore, in some embodiments of the present application, the method further includes:

[0024] Determine a second average value corresponding to the reaction time data at each sleepiness level; wherein the reaction time data is the reaction time data of the experimental target obtained based on the psychomotor vigilance task;

[0025] The calibrated EEG data is cleaned based on the second average value.

[0026] In the above embodiment, when performing data cleaning processing on the EEG data, it is also possible to obtain a second average value corresponding to the reaction time data obtained based on the psychomotor vigilance task at each sleepiness level, and then perform data cleaning on the calibrated EEG data based on the second average value, which can eliminate EEG data that differs greatly from the second average value and improve the reliability of the processed EEG data.

[0027] Furthermore, in some embodiments of the present application, the performing feature extraction processing based on the processed EEG data to obtain the training data set includes:

[0028] Performing single-task data cleaning processing on the processed EEG data obtained under the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test, and the auditory P300 test task to obtain first EEG data;

[0029] Performing feature extraction processing on the first EEG data to obtain feature data corresponding to the first EEG data;

[0030] The feature data are spliced ​​to obtain the training data set.

[0031] In the above embodiment, when performing feature extraction processing on the processed EEG data, it is necessary to first perform targeted single-task data cleaning processing on the EEG data corresponding to each task in the processed EEG data, so as to form the first EEG data according to the EEG data obtained after the single-task data cleaning processing, and then perform feature extraction processing on the first EEG data to extract the feature data corresponding to the first EEG data. Finally, these feature data are spliced ​​to obtain a training data set that can be used to train the model. The initial drowsiness detection model is trained based on this training data set to obtain the trained drowsiness detection model, which can improve the detection accuracy of the trained drowsiness detection model when detecting driver drowsiness.

[0032] Further, in some embodiments of the present application, the collecting of EEG data and eye movement data of the experimental target under the first preset task, the second preset task, and the third preset task includes:

[0033] Collecting EEG data and eye movement data of the experimental subject during the psychomotor vigilance task lasting 10 minutes;

[0034] Collecting EEG data and eye movement data of the experimental subject during a Karolinska Sleepiness Scale task lasting 1 minute;

[0035] Collecting EEG data and eye movement data of the experimental target under the Karolinska sleepiness test task in the first state and the alpha decay test task in the first state, which are carried out simultaneously and last for 5 minutes;

[0036] Collecting EEG data and eye movement data of the experimental subject under the Karolinska sleepiness test task in the second state and the alpha decay test task in the second state, which are performed simultaneously and last for 2 minutes; wherein the second state represents that the experimental subject is in a closed-eye state;

[0037] Collecting EEG data and eye movement data of the experimental subject in the alpha decay test task in the first state lasting 2 minutes;

[0038] Collecting EEG data and eye movement data of the experimental subject in the alpha decay test task in the second state lasting for 2 minutes;

[0039] Collecting EEG data and eye movement data of the experimental subject under the visual P300 test task lasting 2 minutes;

[0040] Collecting EEG data and eye movement data of the experimental subject in the auditory P300 test task lasting 2 minutes;

[0041] The EEG data and eye movement data of the experimental subject were collected during the visual analog scale task lasting 4 minutes.

[0042] In the above embodiment, each experimental task is executed in a certain execution order, and each task has a corresponding duration during the execution process. Therefore, when obtaining the EEG data and eye movement data of the experimental target throughout the experiment, the EEG data and eye movement data of the experimental target under the psychomotor vigilance task lasting 10 minutes can be first collected; then the EEG data and eye movement data of the experimental target under the Karolinska Sleepiness Scale task lasting 1 minute can be collected; then the EEG data and eye movement data of the experimental target under the Karolinska sleepiness test task in the open eyes state and the alpha decay test task in the open eyes state, which are carried out simultaneously and last for 5 minutes, are collected; then the EEG data and eye movement data of the experimental target under the Karolinska sleepiness test task in the closed eyes state and the alpha decay test task in the closed eyes state, which are carried out simultaneously and last for 2 minutes, are collected. Collect the EEG data and eye movement data of the experimental target under the alpha decay test task in the eyes-open state for 2 minutes; then collect the EEG data and eye movement data of the experimental target under the alpha decay test task in the second state for 2 minutes; then collect the EEG data and eye movement data of the experimental target under the visual P300 test task for 2 minutes; then collect the EEG data and eye movement data of the experimental target under the auditory P300 test task for 2 minutes; then collect the EEG data and eye movement data of the experimental target under the 100 mm visual analog scale task for 4 minutes; thus complete the collection of the EEG data and eye movement data of the entire designed experiment, and obtain the EEG data and eye movement data based on various designed experimental tasks. Subsequent model training is carried out based on these data, which is conducive to improving the detection accuracy of the trained sleepiness detection model.

[0043] Further, in some embodiments of the present application, the first-state Karolinska sleepiness test task and the first-state alpha decay test task are test tasks performed based on static image stimulation in which the experimental target of the first state is in a static cockpit;

[0044] The second-state Karolinska sleepiness test task and the second-state alpha decay test task are test tasks that the experimental subject performs in the second state for 2 minutes in a static cockpit;

[0045] The visual P300 test task is a test task based on dynamic image stimulation;

[0046] The auditory P300 test task is a test task based on dynamic sound stimulation.

[0047] In the above embodiments, the Karolinska sleepiness test task and the alpha decay test task in the eyes-open state refer to test tasks completed by the experimental target in a static cockpit based on static image stimulation; the Karolinska sleepiness test task and the alpha decay test task in the eyes-closed state refer to test tasks completed by the experimental target in a static cockpit with eyes closed for 2 minutes; the visual P300 test task refers to a test task completed by the experimental target under dynamic image stimulation; the auditory P300 test task refers to a test task completed by the experimental target under dynamic sound stimulation; various experiments completed under different states and different stimuli based on the above experimental targets can simulate various stimuli and possible driving states that the driver may be subjected to in daily driving scenarios, so that the obtained EEG data is closer to the real driving scenario, which is conducive to improving the detection accuracy of the driver's drowsiness in the real driving scenario.

[0048] In a second aspect, an embodiment of the present application provides an electronic device, the electronic device comprising an acquisition unit and a detection unit,

[0049] The acquisition unit is used to collect the EEG data of the driver when driving;

[0050] The detection unit is used to perform drowsiness detection on the EEG data using the trained drowsiness detection model to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task;

[0051] The preset experimental tasks include a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the sleepiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

[0052] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory storing instructions executable by the processor; when the instructions are executed by the processor, the above-mentioned driver drowsiness detection method is implemented.

[0053] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, which is applied to an electronic device, and when the program is executed by a processor, the above-mentioned driver drowsiness detection method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 A schematic diagram of the implementation process of the driver drowsiness detection method proposed in the embodiment of the present application;

[0055] Figure 2 A schematic diagram of the experimental tasks proposed for the embodiments of the present application;

[0056] Figure 3 A schematic diagram of an implementation of a driver drowsiness detection method proposed in an embodiment of the present application;

[0057] Figure 4 Schematic diagram of the composition structure of the electronic device proposed in the embodiment of the present application Figure 1 ;

[0058] Figure 5 Schematic diagram of the composition structure of the electronic device proposed in the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0059] The following will describe the implementation methods of the present application with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0060] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.

[0061] The driver's drowsiness directly affects his driving safety; when the driver is in a drowsy state, he should be reminded and advised to get enough rest. When detecting and analyzing the physiological state of drowsiness, data such as the driver's external behavior are currently often used. This method has the problem of not being direct and objective. Among the relevant drowsiness detection methods, most are non-contact detection, that is, the detection method does not require the driver to wear any sensors that contact the body. The advantage of this method is that it is easy to use and does not require any additional operation from the driver, but this method is easily affected by various factors and there are many inconveniences when using it. For example, the camera for detecting drowsiness is greatly affected by light, or some drivers can enter a micro-sleep state with their eyelids open normally, and the driver cannot wear sunglasses, masks or other obstructions during the detection process; in this case, contact detection shows obvious advantages, that is, the driver needs to wear sensors that contact the body to collect physiological signals.

[0062] However, most of the current contact detection systems are based on the driver's eyes and body posture for detection, which is easily affected by the environment and individual differences, and will affect the detection accuracy. Due to the different physiological characteristics and habits of different drivers, some drivers may not be accurately detected and cannot be applied to all drivers; in addition, some drivers can enter a micro-sleep state when their eyelids are normally open, which will seriously lead to inaccurate feature judgment; and for other drivers who are only in a closed-eye state, it is necessary to calibrate their drowsiness in advance, which is time-consuming and inefficient; and currently, the threshold is often combined to determine whether the driver is tired. The drowsiness state may have different manifestations for different drivers. Therefore, the setting of the threshold may not accurately reflect the driver's drowsiness, resulting in a decrease in detection accuracy. Moreover, due to the large individual differences among drivers, the drowsiness state of some drivers may not be accurately judged by the preset threshold, which will also reduce the detection accuracy. The threshold detection method can only consider one or a few physiological indicators, while ignoring other factors that may affect the driver's drowsiness state, such as driving conditions, driving style, etc. In other words, how to improve the detection accuracy of the driver's drowsiness is a technical problem that needs to be solved urgently.

[0063] In order to solve the above problems, an embodiment of the present application provides a driver drowsiness detection method, an electronic device and a storage medium, wherein the electronic device collects EEG data of the driver in a driving state; uses a trained drowsiness detection model to perform drowsiness detection on the EEG data to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; the preset experimental task includes a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the drowsiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task; thereby, accurate detection of the driver's drowsiness can be achieved.

[0064] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0065] The present application provides a method for detecting driver drowsiness. Figure 1 This is a schematic diagram of the implementation process of the driver drowsiness detection method proposed in the embodiment of the present application, such as Figure 1 As shown, the driver drowsiness detection method of the electronic device may include the following steps:

[0066] Step 101: Collect the EEG data of the driver while driving.

[0067] In an embodiment of the present application, the electronic device may first collect the driver's EEG data while driving.

[0068] It should be noted that in the embodiments of the present application, EEG is the most intuitive indicator signal of the brain's drowsiness state. Based on brain-computer interface technology, accurate drowsiness detection can be achieved, thereby improving driving safety; the driver's EEG data is obtained based on brain-computer interface technology, and the EEG data can directly reflect the driver's real physiological characteristics in different drowsiness states, thereby accurately detecting the driver's drowsiness based on the EEG data.

[0069] Step 102: Perform drowsiness detection on the EEG data using the trained drowsiness detection model to obtain the drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training the initial drowsiness detection model using a training data set obtained based on a preset experimental task; the preset experimental task includes a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the drowsiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

[0070] In an embodiment of the present application, after collecting the EEG data of the driver in a driving state, the electronic device can use the trained drowsiness detection model to perform drowsiness detection on the EEG data to obtain the driver's drowsiness detection result; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; the preset experimental task includes a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the drowsiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

[0071] It should be noted that, in the embodiments of the present application, the initial sleepiness detection model may be a support vector machine (SVM) model, and thus the trained sleepiness detection model is obtained by training the SVM model using a training data set.

[0072] It should be noted that, in the embodiments of the present application, static stimulation refers to stimulation that is static and does not change. For example, the experimental target observes an image that does not change, which is a static stimulation; dynamic stimulation refers to stimulation that is dynamic and will change. For example, the experimental target observes an image that keeps changing on an electronic screen, or different images that appear randomly, which is a dynamic stimulation.

[0073] It should be noted that, in the embodiment of the present application, the third preset task is used to determine the sleepiness level, which can be a subjective and objective sleepiness level; for example, the subjective scale task includes the Karolinska Sleepiness Scale task and the visual analog scale task; these subjective scale tasks refer to tasks in which the experimental target is given a sleepiness level according to its sleepiness condition.

[0074] It should also be noted that, in the embodiments of the present application, the training data set is obtained based on the experimental objectives and the first preset task, the second preset task and the third preset task designed above; the experimental objectives refer to the measured targets or measured persons of the above-mentioned first preset task, the second preset task and the third preset task; the number of experimental targets is not limited in this application.

[0075] Furthermore, in an embodiment of the present application, the drowsiness detection result can characterize the drowsiness level of the driver in the current driving state, that is, the drowsiness detection result can accurately reflect the driver's current drowsiness level, thereby achieving accurate detection of the driver's drowsiness.

[0076] Furthermore, in an embodiment of the present application, the Karolinska sleepiness test task is a test task for evaluating whether there is a tendency to sleep too much; and the Karolinska Sleepiness Scale (KSS) task can be used to measure the sleepiness of the experimental target. Exemplarily, the present application can use the Chinese Karolinska Sleepiness Scale (KSS-CN) to measure the sleepiness of the experimental target. KSS-CN includes different levels: 1 point for very alert, 3 points for alert, 5 points for neither alert nor sleepy, 7 points for sleepy (but not resistant to sleep), and 9 points for very sleepy (resistant to sleep).

[0077] Further, in an embodiment of the present application, the Alpha Attenuation Test (AAT) is a specific EEG analysis method used to measure the alpha wave power ratio between eyes open and eyes closed states; in the AAT task, electrodes are placed on the scalp of the experimental subject to record the electrical activity of the brain, and the experimental subject is asked to close his eyes for a period of time, and open his eyes for the same period of time to determine the alpha wave power ratio.

[0078] Furthermore, in an embodiment of the present application, the visual P300 test task is a neurophysiological measurement task for evaluating the brain's attention and cognitive functions, and the test task is based on the "P300" event-related potentials (ERPs) generated by the brain under specific visual stimulation conditions; and the auditory P300 test task is similar to the visual P300 test task, and is a test task completed based on the "P300" event-related potentials (ERPs) generated by the brain under specific auditory stimulation conditions.

[0079] Furthermore, in an embodiment of the present application, a psychomotor vigilance task (PVT) can measure an individual's attention level and alertness based on his or her reaction time.

[0080] Furthermore, in an embodiment of the present application, the visual analog scale is a subjective assessment tool, and the visual analog scale in the present application can adopt a 100 mm visual analog scale; the 100 mm visual analog scale task can be used to quantify the degree or intensity of an individual's feeling or perception.

[0081] In some embodiments of the present application, when acquiring a training data set, EEG data and eye movement data of the experimental target under a first preset task, a second preset task, and a third preset task can be collected to obtain a first data set; the EEG data under the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test task, and the auditory P300 test task in the first data set are calibrated for sleepiness grading according to the subjective scale data of the experimental target under the Karolinska sleepiness scale task and the visual analog scale task to obtain calibrated EEG data; data cleaning processing is performed on the calibrated EEG data based on the eye movement data in the first data set to obtain processed EEG data; feature extraction processing is performed based on the processed EEG data to obtain a training data set.

[0082] It should be noted that in the embodiments of the present application, the experimental subject can wear a 64-lead wireless EEG acquisition device and an eye tracker during the entire process of participating in the first preset task, the second preset task, and the third preset task; for example, before performing the experimental task, the experimental subject can first carry out a period of preparation work, such as 30 minutes of preparation time, mainly to wear the EEG acquisition device and the eye tracker, etc.; thereby, the electronic device can form a first data set based on the EEG data and eye movement data of the experimental subject obtained throughout the experiment.

[0083] In some embodiments of the present application, when collecting EEG data and eye movement data of the experimental target under the first preset task, the second preset task and the third preset task, the EEG data and eye movement data of the experimental target under the psychomotor vigilance task lasting 10 minutes can be collected; the EEG data and eye movement data of the experimental target under the Karolinska Sleepiness Scale task lasting 1 minute can be collected; the EEG data and eye movement data of the experimental target under the first state Karolinska sleepiness test task and the first state alpha decay test task, which are carried out simultaneously and last for 5 minutes, are collected; the EEG data and eye movement data of the experimental target under the second state Karolinska sleepiness test task and the second state alpha decay test task, which are carried out simultaneously and last for 2 minutes, are collected. The EEG data and eye movement data of the experimental target under the alpha decay test task lasting 2 minutes were collected; the second state represents that the experimental target is in the eyes-closed state; the EEG data and eye movement data of the experimental target under the alpha decay test task in the first state lasting 2 minutes; the EEG data and eye movement data of the experimental target under the alpha decay test task in the second state lasting 2 minutes; the EEG data and eye movement data of the experimental target under the visual P300 test task lasting 2 minutes; the EEG data and eye movement data of the experimental target under the auditory P300 test task lasting 2 minutes; the EEG data and eye movement data of the experimental target under the 100 mm visual analog scale task lasting 4 minutes.

[0084] It should be noted that, in the embodiments of the present application, the Karolinska sleepiness test task in the first state and the alpha decay test task in the first state are test tasks performed in which the experimental target is in a static cockpit and based on static image stimulation; the Karolinska sleepiness test task in the second state and the alpha decay test task in the second state are test tasks performed in which the experimental target is in a static cockpit in the second state lasting 2 minutes; the visual P300 test task is a test task performed in which the experimental target is based on dynamic image stimulation; and the auditory P300 test task is a test task performed in which the experimental target is based on dynamic sound stimulation.

[0085] For example, Figure 2 A schematic diagram of the experimental task proposed in the embodiment of the present application, such as Figure 2As shown in FIG. 1 , the process of collecting EEG data and eye movement data of the experimental target under the first preset task, the second preset task and the third preset task can be as follows: first, the EEG data and eye movement data of the experimental target under the psychomotor vigilance task (PVT) lasting 10 minutes are collected, wherein the psychomotor vigilance task uses a simple visual reaction time (RT) paradigm with a stimulation interval ranging from 2 to 10 seconds; the collected performance indicators include: the average value of RTs, the fastest 10% RTs, the slowest 10% RTs, the average value of ... RTs, median RTs, and failure RTs longer than 500ms; then collect the EEG data and eye movement data of the experimental target under the Chinese Karolinska Sleepiness Scale (KSS-CN) task lasting 1 minute; the Chinese Karolinska Sleepiness Scale translated from Chinese was used to measure the sleepiness of the experimental target. KSS-CN contains nine levels: 1 point is very alert, 3 points is alert, 5 points is neither alert nor sleepy, 7 points is sleepy (but not sleep-resistant), and 9 points is very sleepy (sleep-resistant); then collect the Karolinska sleepiness test task and alpha decay test data of the experimental target in the open-eye state for 5 minutes. The EEG data and eye movement data of the AAT task were collected. Since the experimental operations of the Karolinska sleepiness test task and the alpha decay test task are basically the same, they only differ in time and subsequent processing methods, so they can be carried out together; and during the 5-minute Karolinska sleepiness test task and the alpha decay test task, the experimental subject was required to sit in a static cockpit and stare at a picture on the electronic screen (static image stimulus); then the EEG data and eye movement data of the Karolinska sleepiness test task and the alpha decay test task were collected for 2 minutes with the experimental subject in the closed eye state. Eye movement data, during this process, the experimental target was required to sit in a static cockpit with eyes closed for 2 minutes; then the EEG data and eye movement data of the experimental target were collected for the alpha decay test task with eyes open for 2 minutes and the alpha decay test task with eyes closed for 2 minutes, where the eyes open and eyes closed were 2 minutes each, and the experimental target stared at a picture on the electronic screen during this process; then the EEG data and eye movement data of the experimental target were collected for the visual P300 test (V-P300) task lasting 2 minutes, during which the experimental target sat in a quiet The experimental subject sat on a chair in a quiet room and stared at the screen in front of him. A red circle or a green circle could appear randomly on the screen every 0.75s and last for 200ms as visual stimulation (dynamic image stimulation). The red circle was the target stimulus, accounting for about 10% of the total stimulation and appearing 200 times in total. The experimental subject was required to say the number of times the red circle appeared after the experimental task. Then, the EEG data and eye movement data of the experimental subject were collected under the auditory P300 test (A-P300) task lasting 2 minutes. During this process, the experimental subject sat on a chair in a quiet room and listened carefully to the sound played by the speaker in front of him. The speaker could appear every 0.75s and last for 200ms. The red circle was the target stimulus, accounting for about 10% of the total stimulation and appearing 200 times in total. The experimental subject was required to say the number of times the red circle appeared after the experimental task. Then, the EEG data and eye movement data of the experimental subject were collected under the auditory P300 test (A-P300) task lasting 2 minutes. During this process, the experimental subject sat on a chair in a quiet room and listened carefully to the sound played by the speaker in front of him. The speaker could appear every 0.The horns of cars and trucks (higher discrimination) were randomly played for 500 milliseconds as sound stimulation (dynamic sound stimulation) for 75 seconds. The truck horns were the target stimulation, accounting for about 10% of the total stimulation, and appeared 200 times in total. The experimental subjects were required to say the number of times the truck horns were honked after the experiment. Finally, the EEG data and eye movement data of the experimental subjects were collected under the 100 mm visual analog scale task for 4 minutes. This process was equivalent to the final rest stage. After completing the previous test tasks, in the last 4 minutes, the experimental subjects were required to do a 100 mm visual analog scale (VAS). VAS is a 100 mm line, with "not sleepy" at the left end of the line and "sleepy" at the right end of the line. The experimental subjects were required to regard this line as representing the range of their personal feelings and make a mark on this line to indicate their feelings at the time. .

[0086] It should be noted that, in the embodiments of the present application, the 10-minute psychomotor vigilance task, the 1-minute Chinese Karolinska Sleepiness Scale task, the 5-minute Karolinska sleepiness test task and the alpha decay test task performed with the experimental target in the eyes open state, the 2-minute Karolinska sleepiness test task and the alpha decay test task performed with the experimental target in the eyes closed state, the 2-minute alpha decay test task performed with the experimental target in the eyes open state and the 2-minute alpha decay test task performed with the experimental target in the eyes closed state are all static test tasks, and the 2-minute visual P300 test task and the 2-minute auditory P300 test task are all dynamic test tasks.

[0087] Furthermore, in an embodiment of the present application, after completing the above-mentioned experimental tasks and obtaining the EEG data and eye movement data of the entire experiment to obtain the first data set, the EEG data therein can be calibrated. Specifically, during the calibration, the subjective scale data under the Karolinska Sleepiness Scale task and the Visual Analog Scale task can be used to calibrate the sleepiness grade of the EEG data under the Karolinska Sleepiness Test Task, Alpha Decay Test Task, Visual P300 Test Task and Auditory P300 Test Task in the first data set according to the experimental objectives to obtain the calibrated EEG data.

[0088] It can be understood that in the embodiments of the present application, the subjective scale data refers to the subjective scale data given by the experimental subject when performing the Karolinska Sleepiness Scale task and the Visual Analogue Scale task, which can reflect the degree of sleepiness or sleepiness of the experimental subject subjectively.

[0089] Exemplarily, subjective scale data is used to calibrate the sleepiness level of EEG data under the Karolinska sleepiness test task, alpha decay test task, visual P300 test task and auditory P300 test task to obtain sleepiness level labels corresponding to the EEG data, wherein the grading granularity can be adjusted according to actual needs to obtain calibrated EEG data.

[0090] Furthermore, in an embodiment of the present application, after obtaining the calibrated EEG data, it is necessary to perform data cleaning processing on the calibrated EEG data.

[0091] In some embodiments of the present application, the electronic device can perform data cleaning processing on the calibrated EEG data based on the eye movement data in the first data set. Specifically, the target eye movement data can be determined according to the eye movement data in the first data set; wherein the target eye movement data include eye movement data under a psychomotor vigilance task, a subjective scale task, a first-state Karolinska sleepiness test task, and an alpha decay test task; the first state represents that the experimental target is in an open-eye state; the target eye movement data is calibrated for sleepiness grading according to the subjective scale data to obtain calibrated eye movement data; a first average value corresponding to the eye movement data at each sleepiness level in the calibrated eye movement data is calculated; wherein the first average value includes an average value calculated based on the eyelid closure data in the eye movement data; and the calibrated EEG data is data cleaned based on the first average value.

[0092] In some embodiments of the present application, in the process of data cleaning processing of the calibrated EEG data, data loss and data quality problems in the calibrated EEG data can be eliminated first, and then the calibrated EEG data can be cleaned using the target eye movement data.

[0093] Exemplarily, eye movement data based on a psychomotor vigilance task, a subjective scale task, a Karolinska sleepiness test task in an open eyes state, and an alpha decay test task in the first data set can be extracted as target eye movement data, that is, the target eye movement data are regarded as valid eye movement data, and other eye movement data other than the target eye movement data are eliminated; then, the target eye movement data are calibrated for sleepiness grading according to the subjective scale data to obtain a sleepiness grading label corresponding to the target eye movement data, thereby obtaining the calibrated eye movement data according to the target eye movement data and the sleepiness grading label corresponding to the target eye movement data; then, the average value corresponding to the eyelid closure degree (Perclos) data of the eye movement data at each level of sleepiness is calculated, that is, the first average value, thereby eliminating the overall experimental data that is significantly different from the first average value of each sleepiness level, including the EEG data in the calibrated EEG data that is significantly different from the first average value of each sleepiness level, to obtain the processed EEG data.

[0094] In some embodiments of the present application, a second average value corresponding to the reaction time data at each sleepiness level can also be determined; wherein the reaction time data is the reaction time data of the experimental target obtained based on the psychomotor vigilance task; and the calibrated EEG data is cleaned based on the second average value.

[0095] Exemplarily, while the target eye movement data are calibrated for sleepiness levels according to the subjective scale data, the reaction time data of the experimental target obtained based on the psychomotor vigilance task can also be calibrated for sleepiness levels according to the subjective scale data, and after calibration, the second average value of the reaction time data at each sleepiness level is obtained, thereby eliminating the overall experimental data that differs greatly from the second average value level of each sleepiness level, including the EEG data in the calibrated EEG data that differs greatly from the second average value level of each sleepiness level, thereby obtaining processed EEG data based on the EEG data obtained after the above-mentioned elimination of the first average value and the second average value.

[0096] In some embodiments of the present application, when performing feature extraction processing based on the processed EEG data to obtain a training data set, single-task data cleaning processing can be performed on the EEG data obtained under the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test task and the auditory P300 test task in the processed EEG data to obtain first EEG data; feature extraction processing can be performed on the first EEG data to obtain feature data corresponding to the first EEG data; and the feature data can be spliced ​​to obtain a training data set.

[0097] It should be noted that, in the embodiments of the present application, in addition to the single-task data cleaning processing, the abnormal or unreasonable data in the EEG data corresponding to each of the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test task and the auditory P300 test task are eliminated.

[0098] For example, when single-task data cleaning is performed on the processed EEG data obtained under the Karolinska sleepiness test task, the EEG data corresponding to the Karolinska sleepiness test task in the first state (eyes open state) and the second state (eyes closed state) can be firstly analyzed and processed for energy changes of all leads and all rhythms to obtain a first brain topography map, and the abnormal data of the EEG data obtained under the Karolinska sleepiness test task are eliminated according to the first brain topography map; in the processed EEG data, the EEG data obtained under the alpha decay test task can be When single-task data cleaning is performed on EEG data, the energy changes of all leads in the alpha frequency band in the eyes-open state and the eyes-closed state in the EEG data obtained under the alpha decay test task can be analyzed and processed first to obtain the second brain topography, and the abnormal data of the EEG data obtained under the alpha decay test task can be eliminated according to the second brain topography; when single-task data cleaning is performed on the EEG data obtained under the visual P300 test task in the processed EEG data, the waveform of the EEG data obtained under the visual P300 test task can be first peak analyzed. The processing is performed by performing a peak analysis to obtain the time information of the first peak and the peak value information of the first wave, and unreasonable EEG data are eliminated from the EEG data obtained under the visual P300 test task according to the time information of the first peak and the peak value information of the first wave; when the EEG data obtained under the auditory P300 test task in the processed EEG data is subjected to single-task data cleaning processing, the waveform of the EEG data obtained under the auditory P300 test task can be first subjected to a peak analysis processing to obtain the time information of the second peak and the peak value information of the second wave, and unreasonable EEG data are eliminated from the EEG data obtained under the auditory P300 test task according to the time information of the second peak and the peak value information of the second wave; thereby, the single-task data cleaning processing is completed, and the first EEG data is obtained according to the EEG data corresponding to the Karolinska sleepiness test task after the single-task data cleaning processing, the EEG data corresponding to the alpha decay test task after the single-task data cleaning processing, the EEG data corresponding to the visual P300 test task after the single-task data cleaning processing, and the EEG data corresponding to the auditory P300 test task after the single-task data cleaning processing.

[0099] That is to say, in the embodiment of the present application, when performing data cleaning processing, the main thing is to eliminate data loss and data quality problems, especially the data with subjective understanding deviation of the driver, which is mainly manifested as data with large deviations from other behavioral data such as eye movements; then the abnormal data in a single task can be eliminated, which is mainly manifested as data that is significantly different from the frequency domain energy map or time domain waveform map of the EEG data under the same level of drowsiness.

[0100] Furthermore, in an embodiment of the present application, when performing feature extraction processing, the EEG data can be subjected to frequency domain analysis based on fast Fourier transform to obtain the average brain topography under multiple time windows (2s) of each data segment as the frequency domain feature; at the same time, ERP analysis is performed on the EEG data of the visual P300 test task and the auditory P300 test task (only 300ms are viewed) to obtain the time domain feature.

[0101] Exemplarily, when the first EEG data is subjected to feature extraction processing, the EEG data corresponding to the Karolinska sleepiness test task in the first EEG data can be subjected to full-lead and full-rhythm frequency domain feature extraction processing to obtain first feature information; the average value of the EEG data corresponding to the alpha decay test task in the first EEG data is calculated, wherein, since the alpha decay test task is performed multiple times during the entire experiment, the average value of the EEG data of multiple alpha decay test tasks is taken, and the frequency domain features of the alpha frequency band in the eyes-open state (first state) and the eyes-closed state (second state) in the EEG data corresponding to the alpha decay test task, as well as the energy ratio information corresponding to the eyes-open state and the eyes-closed state are extracted to obtain second feature information; the EEG data corresponding to the visual P300 test task in the first EEG data is subjected to waveform latency, amplitude and waveform feature extraction processing to obtain third feature information; the EEG data corresponding to the auditory P300 test task in the first EEG data is subjected to waveform latency, amplitude and waveform feature extraction processing to obtain fourth feature information.

[0102] Exemplarily, when the feature data is spliced ​​to obtain the training data set, the first feature information, the second feature information, the third feature information and the fourth feature information may be spliced ​​to obtain the training data set.

[0103] Furthermore, in an embodiment of the present application, the training data set may be input into an SVM model (initial sleepiness detection model) for training to obtain a trained sleepiness detection model.

[0104] Furthermore, in an embodiment of the present application, when the trained drowsiness detection model is actually applied, that is, when the trained drowsiness detection model is used to perform drowsiness detection on EEG data to obtain the driver's drowsiness detection result, the driver's EEG data can be preprocessed first, and then the preprocessed EEG data can be feature extracted to obtain the EEG data to be processed, and then the EEG data to be processed can be input into the trained drowsiness detection model to obtain the driver's drowsiness detection result; wherein, the preprocessing may include operations such as data clearing processing of the driver's EEG data.

[0105] In summary, in the embodiments of the present application, Figure 3As shown, when obtaining the trained sleepiness detection model, the experimental data can be collected to obtain the first data set; wherein the experimental data are the EEG data and eye movement data of the experimental target obtained through the first preset task, the second preset task and the third preset task; after the experimental data are collected, the EEG data under the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test and the auditory P300 test task in the first data set can be calibrated for sleepiness grading to obtain calibrated EEG data; after the sleepiness grading calibration, the data cleaning process can be performed to obtain the processed EEG data, and then the processed EEG data can be subjected to feature extraction processing to obtain the training data set, and based on The SVM model is trained according to the training data set to obtain a trained drowsiness detection model, and then the trained drowsiness detection model can be tested to generate a model evaluation result, etc. After obtaining the trained drowsiness detection model, the trained drowsiness detection model can be used to perform real-time drowsiness detection on the driver's EEG data during driving. For example, in the application process, after collecting the EEG data of the driver in a driving state, the EEG data can be preprocessed and feature extracted to obtain the EEG data to be processed, and then the EEG data to be processed is input into the trained drowsiness detection model to obtain the drowsiness detection result, wherein the preprocessing can include data cleaning processing of the EEG data.

[0106] It can be seen that in the embodiments of the present application, by designing experimental tasks suitable for monitoring the multi-level drowsiness of the driver, the static conditions in daily driving conditions and the dynamic conditions when subjected to visual and auditory stimulation are simulated, so that the acquired EEG data is closer to the real scene, which is beneficial to improving the accuracy of actual detection; and, based on the subjective scale data, the multi-level drowsiness labels are determined, and the overall EEG data is cleaned by combining subjective and objective methods, which effectively improves the accuracy of the labels; in addition, the experimental data is analyzed and calculated according to the characteristics of the experimental tasks to eliminate abnormal data and then extract feature data, thereby improving the accuracy of the trained drowsiness detection model while ensuring data quality.

[0107] In an embodiment of the present application, an electronic device can collect EEG data of a driver in a driving state; use a trained drowsiness detection model to perform drowsiness detection on the EEG data to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; the preset experimental task includes a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the drowsiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task. It can be seen that the present application obtains a training data set using a first preset task performed under static stimulation, a second preset task performed under dynamic stimulation, and a third preset task for determining the drowsiness level, and the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task; these tasks can fully simulate various types of stimulation to which the driver is subjected in real driving scenarios, thereby obtaining relevant data reflecting the drowsiness of the driver in real driving scenarios, and training the initial drowsiness detection model based on the training data set composed of various types of data that are close to the possible drowsiness in real driving scenarios can effectively improve the detection accuracy of the trained drowsiness detection model for the driver's drowsiness, thereby achieving accurate detection of the driver's drowsiness.

[0108] Based on the above embodiment, in another embodiment of the present application, Figure 4 Schematic diagram of the composition structure of the electronic device proposed in the embodiment of the present application Figure 1 ,like Figure 4 As shown, the electronic device 10 may include an acquisition unit 101 and a detection unit 102 .

[0109] The acquisition unit 101 is used to collect the EEG data of the driver when driving.

[0110] The detection unit 102 is used to perform drowsiness detection on the EEG data using a trained drowsiness detection model to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; the preset experimental task includes a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the drowsiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

[0111] In some embodiments, the acquisition unit 101 is also used to collect the EEG data and eye movement data of the experimental target under the first preset task, the second preset task and the third preset task to obtain a first data set; and to calibrate the sleepiness grade of the EEG data under the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test and the auditory P300 test task in the first data set according to the subjective scale data of the experimental target under the Karolinska sleepiness scale task and the visual analog scale task to obtain calibrated EEG data; and to perform data cleaning processing on the calibrated EEG data based on the eye movement data in the first data set to obtain processed EEG data; and to perform feature extraction processing based on the processed EEG data to obtain the training data set.

[0112] In some embodiments, the acquisition unit 101 is also used to determine target eye movement data based on the eye movement data in the first data set; wherein the target eye movement data includes eye movement data under the psychomotor vigilance task, the subjective scale task, the Karolinska sleepiness test task in the first state, and the alpha decay test task; the first state represents that the experimental target is in an open-eye state; and the target eye movement data is calibrated for sleepiness grade according to the subjective scale data to obtain calibrated eye movement data; and a first average value corresponding to the eye movement data at each sleepiness level in the calibrated eye movement data is calculated; wherein the first average value includes an average value calculated based on the eyelid closure data in the eye movement data; and data cleaning processing is performed on the calibrated EEG data based on the first average value.

[0113] In some embodiments, the acquisition unit 101 is also used to determine a second average value corresponding to the reaction time data at each sleepiness level; wherein the reaction time data is the reaction time data of the experimental target obtained based on the psychomotor vigilance task; and based on the second average value, the calibrated EEG data is cleaned.

[0114] In some embodiments, the acquisition unit 101 is also used to perform single-task data cleaning processing on the processed EEG data obtained under the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test and the auditory P300 test task to obtain first EEG data; and perform feature extraction processing on the first EEG data to obtain feature data corresponding to the first EEG data; and perform splicing processing on the feature data to obtain the training data set.

[0115] In some embodiments, the acquisition unit 101 is further used to collect the EEG data and eye movement data of the experimental target under the psychomotor vigilance task lasting 10 minutes; and to collect the EEG data and eye movement data of the experimental target under the Karolinska Sleepiness Scale task lasting 1 minute; and to collect the EEG data and eye movement data of the experimental target under the Karolinska sleepiness test task in the first state that is performed simultaneously and lasts for 5 minutes and the alpha decay test task in the first state; and to collect the EEG data and eye movement data of the experimental target under the Karolinska sleepiness test task in the second state that is performed simultaneously and lasts for 2 minutes and the alpha decay test task in the second state. Data; wherein, the second state represents that the experimental target is in a closed-eye state; and the EEG data and eye movement data of the experimental target under the alpha decay test task in the first state lasting 2 minutes are collected; and the EEG data and eye movement data of the experimental target under the alpha decay test task in the second state lasting 2 minutes are collected; and the EEG data and eye movement data of the experimental target under the visual P300 test task lasting 2 minutes are collected; and the EEG data and eye movement data of the experimental target under the auditory P300 test task lasting 2 minutes are collected; and the EEG data and eye movement data of the experimental target under the visual analog scale task lasting 4 minutes are collected.

[0116] In some embodiments, the Karolinska sleepiness test task in the first state and the alpha decay test task in the first state are test tasks performed in which the experimental target in the first state is in a static cockpit and is based on static image stimulation; the Karolinska sleepiness test task in the second state and the alpha decay test task in the second state are test tasks performed in which the experimental target is in a static cockpit and is in the second state for 2 minutes; the visual P300 test task is a test task performed in which the experimental target is based on dynamic image stimulation; and the auditory P300 test task is a test task performed in which the experimental target is based on dynamic sound stimulation.

[0117] In the embodiments of the present application, further, Figure 5 Schematic diagram of the composition structure of the electronic device proposed in the embodiment of the present application Figure 2 ,like Figure 5 As shown, the electronic device 11 proposed in the embodiment of the present application may also include a processor 103 and a memory 104 storing executable instructions of the processor 103; further, the electronic device 10 may also include a communication interface 105, and a bus 106 for connecting the processor 103, the memory 104 and the communication interface 105.

[0118] In the embodiment of the present application, the processor 103 can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the function of the processor can also be other, and the embodiment of the present application is not specifically limited. The electronic device 10 can also include a memory 104, which can be connected to the processor 103, wherein the memory 104 is used to store executable program code, the program code includes computer operation instructions, and the memory 104 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0119] In the embodiment of the present application, the bus 106 is used to connect the communication interface 105, the processor 103 and the memory 104, and the mutual communication between these devices.

[0120] In the embodiment of the present application, the memory 104 is used to store instructions and data.

[0121] Further, in an embodiment of the present application, the processor 103 is used to collect EEG data of the driver in a driving state;

[0122] The trained drowsiness detection model is used to perform drowsiness detection on the EEG data to obtain the drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; the preset experimental task includes a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the drowsiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

[0123] In practical applications, the memory 104 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor 103.

[0124] In addition, each functional module in this embodiment can be integrated into one analysis unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.

[0125] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., various media that can store program codes.

[0126] An embodiment of the present application provides an electronic device, which collects EEG data of a driver in a driving state; uses a trained drowsiness detection model to perform drowsiness detection on the EEG data to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; the preset experimental task includes a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the drowsiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task. It can be seen that the present application obtains a training data set using a first preset task performed under static stimulation, a second preset task performed under dynamic stimulation, and a third preset task for determining the drowsiness level, and the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task; these tasks can fully simulate various types of stimulation to which the driver is subjected in real driving scenarios, thereby obtaining relevant data reflecting the drowsiness of the driver in real driving scenarios, and training the initial drowsiness detection model based on the training data set composed of various types of data that are close to the possible drowsiness in real driving scenarios can effectively improve the detection accuracy of the trained drowsiness detection model for the driver's drowsiness, thereby achieving accurate detection of the driver's drowsiness.

[0127] Specifically, the program instructions corresponding to a driver drowsiness detection method in this embodiment can be stored on a storage medium such as a CD, a hard disk, a USB flash drive, etc.; when the program instructions corresponding to a driver drowsiness detection method in the storage medium are read or executed by an electronic device, the following steps are included:

[0128] Collect the driver's EEG data while driving;

[0129] Performing drowsiness detection on the EEG data using the trained drowsiness detection model to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task;

[0130] The preset experimental tasks include a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the sleepiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

[0131] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0132] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

Claims

1. A method for detecting driver drowsiness, characterized in that: The method comprises: Collect the driver's EEG data while driving; Performing drowsiness detection on the EEG data using the trained drowsiness detection model to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; The preset experimental tasks include a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the sleepiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

2. The method according to claim 1, characterized in that The method further comprises: Collecting EEG data and eye movement data of the experimental target under the first preset task, the second preset task, and the third preset task to obtain a first data set; According to the experimental objective, the subjective scale data under the Karolinska Sleepiness Scale task and the Visual Analog Scale task are calibrated for sleepiness grading on the electroencephalogram data under the Karolinska Sleepiness Test task, the Alpha Decay Test task, the Visual P300 test, and the Auditory P300 test task in the first data set to obtain calibrated electroencephalogram data; Performing data cleaning processing on the calibrated EEG data based on the eye movement data in the first data set to obtain processed EEG data; Feature extraction is performed based on the processed EEG data to obtain the training data set.

3. The method according to claim 2, characterized in that The performing data cleaning processing on the calibrated EEG data based on the eye movement data in the first data set includes: Determining target eye movement data according to the eye movement data in the first data set; wherein the target eye movement data includes eye movement data under the psychomotor vigilance task, the subjective scale task, the Karolinska sleepiness test task in the first state, and the alpha decay test task; the first state represents that the experimental target is in an open-eye state; Performing sleepiness grading calibration on the target eye movement data according to the subjective scale data to obtain calibrated eye movement data; Calculating a first average value corresponding to the eye movement data at each sleepiness level in the calibrated eye movement data; wherein the first average value includes an average value calculated based on the eyelid closure degree data in the eye movement data; The calibrated EEG data is cleaned based on the first average value.

4. The method according to claim 3, characterized in that The method further comprises: Determine a second average value corresponding to the reaction time data at each sleepiness level; wherein the reaction time data is the reaction time data of the experimental target obtained based on the psychomotor vigilance task; The calibrated EEG data is cleaned based on the second average value.

5. The method according to any one of claims 2 to 4, characterized in that: The performing feature extraction processing based on the processed EEG data to obtain the training data set includes: Performing single-task data cleaning processing on the processed EEG data obtained under the Karolinska sleepiness test task, the alpha decay test task, the visual P300 test, and the auditory P300 test task to obtain first EEG data; Performing feature extraction processing on the first EEG data to obtain feature data corresponding to the first EEG data; The feature data are spliced ​​to obtain the training data set.

6. The method according to claim 2, characterized in that The collecting of EEG data and eye movement data of the experimental target under the first preset task, the second preset task and the third preset task includes: Collecting EEG data and eye movement data of the experimental subject during the psychomotor vigilance task lasting 10 minutes; Collecting EEG data and eye movement data of the experimental subject during a Karolinska Sleepiness Scale task lasting 1 minute; Collecting EEG data and eye movement data of the experimental target under the Karolinska sleepiness test task in the first state and the alpha decay test task in the first state, which are carried out simultaneously and last for 5 minutes; Collecting EEG data and eye movement data of the experimental subject under the Karolinska sleepiness test task in the second state and the alpha decay test task in the second state, which are performed simultaneously and last for 2 minutes; wherein the second state represents that the experimental subject is in a closed-eye state; Collecting EEG data and eye movement data of the experimental subject in the alpha decay test task in the first state lasting 2 minutes; Collecting EEG data and eye movement data of the experimental subject in the alpha decay test task in the second state lasting for 2 minutes; Collecting EEG data and eye movement data of the experimental subject under the visual P300 test task lasting 2 minutes; Collecting EEG data and eye movement data of the experimental subject in the auditory P300 test task lasting 2 minutes; The EEG data and eye movement data of the experimental subject were collected during the visual analog scale task lasting 4 minutes.

7. The method according to claim 6, characterized in that The Karolinska sleepiness test task in the first state and the alpha decay test task in the first state are test tasks performed based on static image stimulation in which the experimental target in the first state is in a static cockpit; The Karolinska sleepiness test task in the second state and the alpha decay test task in the second state are test tasks performed by the experimental subject in a static cockpit in the second state for 2 minutes; The visual P300 test task is a test task performed by the experimental target based on dynamic image stimulation; The auditory P300 test task is a test task in which the experimental target is based on dynamic sound stimulation.

8. An electronic device, characterized in that: The electronic device comprises an acquisition unit and a detection unit. The acquisition unit is used to collect the EEG data of the driver when driving; The detection unit is used to perform drowsiness detection on the EEG data using the trained drowsiness detection model to obtain a drowsiness detection result of the driver; wherein the drowsiness detection result represents the drowsiness degree of the driver in the current driving state; the trained drowsiness detection model is obtained by training an initial drowsiness detection model using a training data set obtained based on a preset experimental task; The preset experimental tasks include a first preset task for performing under static stimulation, a second preset task for performing under dynamic stimulation, and a third preset task for determining the sleepiness level; the first preset task includes a Karolinska sleepiness test task and an alpha decay test task; the second preset task includes a visual P300 test task and an auditory P300 test task; the third preset task includes a psychomotor vigilance task and a subjective scale task; the subjective scale task includes a Karolinska sleepiness scale task and a visual analog scale task.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory storing instructions executable by the processor; when the instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a program stored thereon, applied to an electronic device, wherein when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images