Fusion risk classification method based on electroencephalogram signals of passengers
By combining early warning signals and P300 signals in high-level autonomous driving systems, a risk classifier based on EEG signals is trained, and the problem of low accuracy of EEG signals risk classification in the existing technology is solved, and the risk is achieved Quick and accurate identification.
Patent Information
- Application Number
- CN202411679330.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-22
- Publication Date
- 2025-02-14
AI Technical Summary
The prior art generally has low accuracy when classification of risk based on occupant EEG signals, especially in high-level autonomous driving scenarios. It is difficult to achieve high-accuracy risk classification simply by relying on P300 signals.
A fusion risk classification method based on occupant EEG signals is proposed. The warning signal feature extraction network is trained through an online learning algorithm, and combined with the P300 signal feature extraction network, the risk classifier based on EEG signals is trained to achieve accurate risk prediction.
By adding data characteristics of early warning signal and realizing online learning of early warning signal, combined with on-board computers, it can achieve rapid and accurate risk identification and improve the accuracy of high-level automatic system risk classification.
Smart Images
Figure CN119441989A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the fields of autonomous driving, artificial intelligence and electroencephalogram (EEG) signal processing, and specifically relates to a fusion risk classification method based on occupant EEG signals. Background Art
[0002] In recent years, many serious traffic accidents have exposed the seriousness of the safety issues of autonomous driving technology. Instantaneous dangerous actions in high-level autonomous driving scenarios can cause significant changes in brain signals and generate P300 signals.
[0003] At present, there are relatively few studies on the application of EEG signals to autonomous driving safety at home and abroad. Most technologies focus on using EEG signals to identify driver fatigue driving. For example, in 2017, RifaiC et al. used sparse belief networks to improve the classification of fatigue driving based on EEG signals; in 2019, LiuY et al. used power spectrum analysis and wavelet analysis to extract 12 EEG signal features for short-term driving status prediction. Fatigue driving and risk classification are two different tasks with completely different application prospects. At present, conventional machine learning algorithms have a generally low accuracy problem when performing risk classification based on occupant EEG signals.
[0004] Usually, 1 to 2 seconds before a dangerous event occurs, the sudden appearance of a target object in the field of vision will cause the occupant's prefrontal brain activity to become active. This signal is called a warning signal. Compared with the previous problem of low accuracy in risk classification based solely on P300 signals, the present invention is aimed at the specific task of risk classification of high-level automatic systems, and combines the unique warning signals under this task to provide a method for improving the accuracy of risk classification of high-level automatic systems based on occupant EEG signals. Summary of the invention
[0005] 1. Technical issues to be resolved
[0006] The technical problem to be solved by the present invention is how to provide a fusion risk classification method based on occupant EEG signals to solve the problem of low accuracy of risk classification based solely on P300 signals.
[0007] (II) Technical solution
[0008] In order to solve the above technical problems, the present invention proposes a fusion risk classification method based on occupant EEG signals, which includes:
[0009] Part 1: Use online learning algorithms to train a warning signal feature extraction network for warning signals;
[0010] The second part combines the features extracted by the warning signal feature extraction network and the features extracted by the P300 signal feature extraction network to realize the training of the risk classifier based on EEG signals;
[0011] Part 3: Accurate prediction of risks based on EEG signals.
[0012] (III) Beneficial effects
[0013] The present invention proposes a fusion risk classification method based on occupant EEG signals. Compared with previous studies that simply rely on P300 signals, the present invention adds warning signal data features and can realize online learning of warning signals. Combined with an on-board computer, it can achieve rapid and accurate identification of risks.
[0014] The present invention provides a method for improving the risk classification of high-level automatic systems based on occupant EEG signals. The method is expected to combine with intelligent decision-making technology to achieve the purpose of using occupant EEG signals to make up for the functional deficiencies and performance limitations of the machine. The method provided by the present invention has great application potential in avoiding autonomous driving accidents caused by the functional deficiencies of artificial intelligence algorithms in terms of robustness, generalization, interpretability, logic, completeness, rule coverage, etc. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 It is the technical roadmap of the present invention. DETAILED DESCRIPTION
[0016] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.
[0017] The present invention mainly relates to a method for improving the accuracy of high-level automatic system risk classification, and specifically relates to a method for improving the accuracy of high-level automatic system risk classification based on occupant electroencephalogram signals.
[0018] This invention mainly aims at the problem of safety accidents caused by the limitations of high-level autonomous driving systems. It combines warning signals and P300 signals to provide a fusion risk classification method based on occupant EEG signals, which is used to improve the accuracy of high-level automatic system risk classification based on occupant EEG signals and realize rapid and accurate risk identification. This technology consists of 3 parts:
[0019] Part 1: Use online learning algorithms to train a warning signal feature extraction network for warning signals;
[0020] The second part combines the features extracted by the warning signal feature extraction network and the features extracted by the P300 signal feature extraction network to realize the training of the risk classifier based on EEG signals;
[0021] Part 3: Accurate prediction of risks based on EEG signals.
[0022] Embodiment 1:
[0023] The technical route of the present invention is as follows Figure 1 As shown in the figure. First, the EEG acquisition system collects the EEG signals of passengers when the self-driving car is driving; then, the collected EEG signals are sent to the embedded system (signal processing device), which is responsible for preprocessing the EEG signals, including filtering, removing physiological artifacts and downsampling; finally, online learning is used to train a warning signal feature extraction network that extracts the signal features of the warning signal in real time, and a risk classifier is trained in combination with offline data to achieve the purpose of improving the accuracy of risk classification based on the EEG signals of the occupants. The fusion risk classification method based on EEG signals includes three modules: online learning of warning signals, offline training of P300 signals, and accurate risk prediction of EEG signals.
[0024] Part I: Online Learning of Early Warning Signals
[0025] S11. First, the one-dimensional EEG signal (EEG signal) from the first 1.5 seconds to 0.5 seconds is transformed into a two-dimensional time-frequency graph;
[0026] S12. Secondly, the two-dimensional time-frequency graph is input into the warning signal feature extraction network model, and the feature extraction is performed using neural network models such as CNN, EEGNet, and ResNet;
[0027] S13. Finally, the initial training-free online ELM is used to realize the online learning of the link weights from the full link layer to the output layer. The update formula at time k is as follows:
[0028]
[0029] Among them, β(k) and β(k-1) represent the link weights from the fully connected layer to the output layer at time k and time k-1 respectively, ρ is the forgetting factor, P(k) and P(k-1) are the intermediate variables at time k and time k-1 respectively, H(k) and H(k-1) are the output values of the fully connected layer at time k and time k-1 respectively, and T(k) is the actual category of the warning signal at time k.
[0030] Part II: Offline Training of P300 Signals
[0031] S21. First, the one-dimensional EEG signal from the previous 0.5 seconds to the current moment is transformed into a two-dimensional time-frequency diagram using Stockwell transformation;
[0032] S22. Secondly, the two-dimensional time-frequency graph is input into the P300 signal feature extraction network model, and the features of the P300 signal are extracted using neural network models such as CNN, EEGNet, and ResNet;
[0033] S23. Finally, the two sets of features extracted from the warning signal and the P300 signal are input into a risk classifier, which is a type of temporal network, such as an RNN network, for offline training of the P300 signal.
[0034] Part 3: Accurate prediction of risks of EEG signals
[0035] Firstly, the EEG signal acquisition system is used to collect the EEG signals of the occupants in the intelligent driving scenario. Secondly, the collected EEG signals are sent to the embedded system for signal preprocessing. Finally, the risk results are obtained by using the fusion risk classification method based on the occupants' EEG signals, so as to achieve the purpose of rapid and accurate risk identification.
[0036] Compared with previous studies that simply relied on P300 signals, the present invention adds warning signal data features and can realize online learning of warning signals. Combined with the on-board computer, it can achieve rapid and accurate identification of risks.
[0037] The present invention provides a method for improving the risk classification of high-level automatic systems based on occupant EEG signals. The method is expected to combine with intelligent decision-making technology to achieve the purpose of using occupant EEG signals to make up for the functional deficiencies and performance limitations of the machine. The method provided by the present invention has great application potential in avoiding autonomous driving accidents caused by the functional deficiencies of artificial intelligence algorithms in terms of robustness, generalization, interpretability, logic, completeness, rule coverage, etc.
[0038] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A fusion risk classification method based on occupant EEG signals, characterized in that: The method includes: Part 1: Use online learning algorithms to train a warning signal feature extraction network for warning signals; The second part combines the features extracted by the warning signal feature extraction network and the features extracted by the P300 signal feature extraction network to realize the training of the risk classifier based on EEG signals; Part 3: Accurate prediction of risks based on EEG signals.
2. The fusion risk classification method based on occupant EEG signals according to claim 1, characterized in that: First, the EEG acquisition system is used to collect the EEG signals of passengers while the self-driving car is driving; Then, the collected EEG signals are sent to the embedded system, which is responsible for preprocessing the EEG signals. Finally, online learning is used to train a warning signal feature extraction network to extract warning signals in real time, and a risk classifier is trained in combination with offline data to achieve the purpose of improving the accuracy of risk classification based on occupant EEG signals.
3. The fusion risk classification method based on occupant EEG signals as claimed in claim 2, characterized in that: Preprocessing includes: Filtering, removal of physiological artifacts, and downsampling.
4. The fusion risk classification method based on occupant EEG signals according to any one of claims 1 to 3, characterized in that: The first part includes: S11. First, the one-dimensional EEG signal from the first 1.5 seconds to 0.5 seconds, i.e., the EEG signal, is converted into a two-dimensional time-frequency graph by using Stockwell transformation; S12, secondly, inputting the two-dimensional time-frequency graph into the warning signal feature extraction network model, and performing feature extraction using the warning signal feature extraction network model; S13. Finally, the initial training-free online extreme learning machine is used to realize the online learning of the link weights from the full link layer to the output layer, and the network parameters for extracting the warning signal features are updated online in real time.
5. The fusion risk classification method based on occupant EEG signals according to claim 4, characterized in that: In step S12, the warning signal feature extraction network model is a CNN, EEGNet or ResNet model.
6. The fusion risk classification method based on occupant EEG signals according to claim 4, characterized in that: When the warning signal feature extraction network parameters are updated online in real time in step S13, the update formula at time k is as follows: Among them, β(k) and β(k-1) represent the link weights from the fully connected layer to the output layer of the warning signal feature extraction network at time k and time k-1, respectively, ρ is the forgetting factor, P(k) and P(k-1) are the intermediate variables at time k and time k-1, respectively, H(k) and H(k-1) are the output values of the fully connected layer at time k and time k-1, respectively, and T(k) is the actual category of the warning signal at time k.
7. The fusion risk classification method based on occupant EEG signals according to claim 4, characterized in that: The second part includes: S21. First, the one-dimensional EEG signal from the previous 0.5 seconds to the current moment is transformed into a two-dimensional time-frequency diagram using Stockwell transformation; S22, secondly, inputting the two-dimensional time-frequency graph into the P300 signal feature extraction network model, and using the P300 signal feature extraction network model to extract features of the P300 signal; S23. Finally, the two sets of features extracted from the warning signal and the P300 signal are input into the risk classifier to perform offline training of the P300 signal.
8. The fusion risk classification method based on occupant EEG signals according to claim 7, characterized in that: In step S22, the neural network model is a CNN, EEGNet or ResNet model.
9. The fusion risk classification method based on occupant EEG signals according to claim 7, characterized in that: In step S23, the risk classifier is a RNN network.
10. The fusion risk classification method based on occupant EEG signals according to claim 7, characterized in that: The third part includes: first, using the EEG signal acquisition system to collect the EEG signals of the occupants in the intelligent driving scenario; second, sending the collected EEG signals to the embedded system for signal preprocessing; finally, using the fusion risk classification method based on the occupants' EEG signals to obtain risk results, so as to achieve the purpose of rapid and accurate risk identification.