Driving behavior classification method based on hybrid neural dynamic encoder

Abstract

The invention discloses a driving behavior classification method based on a hybrid neural dynamic encoder, and relates to the field of intelligent driving assistance and electroencephalogram signal classification. The method mainly comprises the steps of electroencephalogram signal acquisition, hybrid neural dynamic coding and driving behavior classification. Performing wavelet denoising and independent component analysis on the extracted EEG signals to remove motion artifacts, and then generating a preprocessed time-frequency feature matrix; the neural dynamic coding module adopts a deformable deep convolution kernel to extract local frequency domain features of EEG signals and a layered space-time attention mechanism to model spatial relevance between EEG channels and a driving behavior time sequence evolution rule, and outputs a space-time joint feature vector; and the driving behavior classification module is responsible for realizing a function of distinguishing five types of driving behaviors in real time. According to the method, through fusion of a biological neuron mechanism and dynamic coding, the feature expression ability of the model to the non-stationary electroencephalogram signals is remarkably improved, and the electroencephalogram signal classification and recognition effect is further improved.

Description

Technical Field

[0001] The present invention relates to the field of electroencephalogram (EEG) signal classification, and more specifically, to a driving behavior classification method based on a hybrid neural dynamic encoder. Background Art

[0002] Electroencephalogram (EEG) signals are electrical wave changes formed by the summation of postsynaptic potentials synchronously generated by a large number of neurons during brain activity, and are the overall reflection of the electrophysiological activities of brain nerve cells on the cerebral cortex or the scalp surface. People usually place electrodes on the human scalp to detect EEG signals and use related devices for collection and processing.

[0003] With the development of driving behavior analysis technology, the classification method of driving behavior based on electroencephalogram (EEG) signals has been widely used. However, there are still some problems in the existing classification methods in actual use. For example, the classification methods on the market usually adopt traditional signal processing and machine learning technologies, with low classification accuracy and real-time performance, making it difficult to meet the requirements of modern intelligent driving systems for accurate analysis and rapid response. This has led to problems such as low recognition accuracy and slow response speed in some scenarios. To improve the performance, some researchers have tried to introduce deep learning models to improve the classification effect. However, such improvements often face problems such as high model complexity, large computational volume, and the need for a large amount of labeled data, restricting their practical applications. After retrieval, a method for classifying EEG signals based on Vision Transformer with the publication number CN114176607B was disclosed, and the publication date was April 19, 2024. This method can achieve good performance in the EEG signal classification task by data preprocessing, feature extraction, and training an EEG signal classification model based on Vision Transformer. However, this method is mainly applicable to static EEG signals, and for dynamically changing driving behavior signals, its classification accuracy and real-time performance are not ideal. In addition, during the feature extraction and model training processes of this method, the quality requirements for data are relatively high, and it is easily affected by noise and individual differences, resulting in insufficient model generalization ability. After retrieval, a method for classifying EEG signals based on brain functional connectivity network features with the publication number CN116541751B was disclosed, and the publication date was September 12, 2023. This method collects motor imagery EEG signals, performs preprocessing and feature extraction, selects important EEG channels by combining with the brain functional connectivity network, and then uses the SVM model for classification. Although this method utilizes the features of the brain functional connectivity network and improves the classification effect, it mainly targets specific motor imagery tasks, and its adaptability and real-time performance are insufficient for complex and dynamically changing signals such as driving behavior. In addition, during the channel selection and feature extraction processes of this method, the features in the time dimension are not considered, resulting in weak processing ability of the model for time-series changing signals. The above problems indicate that the traditional classification methods on the market are difficult to effectively meet the new requirements of complex driving behavior analysis for high accuracy and rapid response.

[0004] Therefore, a driving behavior classification method based on a hybrid neural dynamic encoder is desired. Summary of the Invention

[0005] The present invention provides a driving behavior classification method based on a hybrid neural dynamic encoder to solve the problems presented in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: A driving behavior classification method based on a hybrid neural dynamic encoder specifically includes the following steps:

[0007] Step S1: EEG signal acquisition and preprocessing

[0008] 1.1 Signal acquisition

[0009] Collect the driver's EEG signals through a wireless EEG sensor (such as NeuroSky MindWave Mobile 2 or OpenBCI device). The sampling rate is set to 256 Hz, and the electrode layout follows the international 10-20 system, focusing on covering the premotor cortex (F3, F4), parietal lobe (P3, P4), and motor-related areas (C3, C4). During the acquisition process, driving behavior event tags (such as braking, lane changing, accelerating, fatigue, distraction) are synchronously recorded, and the timestamp accuracy is 1 ms.

[0010] 1.2 Wavelet denoising

[0011] Use the Morlet wavelet basis function, which is suitable for analyzing non-stationary EEG signals due to its advantages in time-frequency localization characteristics. Perform 6-layer wavelet decomposition on the original signal, retain the frequency band of 4 - 30 Hz (covering θ, α, β rhythms), and remove low-frequency baseline drift (<4 Hz) and high-frequency noise (>30 Hz). Perform threshold processing (hard threshold, threshold is 3σ, σ is the noise standard deviation) on the wavelet coefficients of the retained frequency band, and the reconstructed signal is denoted as

[0012] 1.3 Independent component analysis

[0013] Independent component analysis (ICA) separates the motion artifact components through the FastICA algorithm and calculates the artifact contribution degree. The formula is as follows:

[0014]

[0015] where i is the index variable used to traverse the artifact part, j is the index variable used to traverse all components, k is the number of artifact components, n is the total number of components, and when η > 0.3, signal re-acquisition is triggered.

[0016] 1.4 Dynamic parameter initialization

[0017] Calculate the individual frequency band energy mean μ based on the driver's baseline EEG signal (resting state) band , initialize the bias term b of the convolutional layer init = μ band · W b , to eliminate the influence of individual physiological differences. Generate the preprocessed time-frequency feature matrix Μ ∈ R C×T , where C = 8 (number of key channels), T = 128 (time window length 0.5 seconds).

[0018] Step S2: Utilize hybrid neural dynamic coding

[0019] 2.1 Multi-scale frequency domain decomposition unit: The deformable depth convolution kernel (Deformable DWConv) is used to dynamically adapt to the time-frequency distribution of the signal. Through the formula:

[0020]

[0021] The optimal kernel size K∈{3,5,7} is selected. Among them, Ψ(X) is the wavelet packet energy distribution, and W k is the convolution kernel weight matrix of size k, and X is the time-frequency feature matrix obtained after preprocessing the input EEG signal.

[0022] 2.2 Hierarchical spatio-temporal attention mechanism

[0023] Spiking neural network temporal encoder: Based on the LIF neuron model, the features are spiking encoded, and the membrane potential update formula is:

[0024]

[0025] Among them, τ is the decay coefficient, and V th is the triggering threshold, Θ is the step function, represents the membrane potential of the neuron at time t, represents the value of the jth input signal.

[0026] 2.3 Spatio-temporal joint encoding module:

[0027] On the spatial path, a graph attention network (GAT) is used to generate the brain region functional connectivity matrix:

[0028] A dynamic = GAT(E node ||E time )

[0029] E node represents the node feature intervention, and E time represents the time feature embedding;

[0030] On the time path, squeeze-and-excitation temporal convolution (SE-Conv) is used to capture the evolution law of the behavior intention through the formula. The formula is as follows:

[0031] F out = σ(W C ·GlobalAvgPool(F in ))⊙F in

[0032] Among them, F in represents the feature map input to the squeeze-and-excitation temporal convolution module, and W Cis a learnable weight matrix.

[0033] Step S3: Driving behavior classification

[0034] 3.1 Softmax classifier

[0035] Input h st to the fully connected layer (dimension 256→128→5), and the activation function is ReLU.

[0036] Output the probability distribution of five types of driving behaviors: P = Softmax(W f h st +b f ) where h st is the spatio-temporal joint feature vector, and b f is a bias vector of length 5.

[0037] 3.2 Model quantization and compression

[0038] Post-training quantization (PTQ): Quantize the model parameters from FP32 to INT8, adopt the symmetric quantization strategy, and the calibration dataset is 1000 randomly sampled EEG segments.

[0039] Performance metrics: The model volume is compressed to 0.9MB, the inference latency ≤ 10ms (based on the ARM Cortex-M7 microcontroller), and the classification accuracy degradation ≤ 1.5%.

[0040] 3.3 Real-time deployment optimization

[0041] Memory allocation: Adopt the static memory pre-allocation strategy to avoid dynamic memory fragmentation.

[0042] Multi-threaded pipeline: Signal acquisition, preprocessing, and inference are executed in separate threads, and the frame processing cycle ≤ 50ms.

[0043] An electronic device, including a memory, a processor, and a computer program stored on the memory, where the processor implements the driving behavior classification method based on the hybrid neural dynamic encoder when executing the program.

[0044] A computer-readable storage medium stores a computer program, and the program implements the driving behavior classification method based on the hybrid neural dynamic encoder when executed by the processor.

[0045] Compared with the prior art, the advantages of the present invention are:

[0046] (1) In terms of method innovation, the present invention combines the LIF pulsed neural network with deformable depth convolution kernels for the first time, and realizes brain-like pulsed coding of features through dynamic update of membrane potential, reducing redundant information by 80% compared with traditional continuous-valued features. A dual-channel attention mechanism (dynamic graph convolution + squeeze-and-excitation temporal convolution) is constructed to dynamically capture the functional connectivity of brain regions (A dynamic = GAT(E node ||E time )) and the temporal evolution law (F out = σ(W C ·GlobalAvgPool(F in )) ⊙ F in ), improving the spatio-temporal feature expression ability compared with traditional CNN-LSTM.

[0047] (2) In terms of performance, the number of model parameters can be further compressed. Through pulsed coding and model quantization (INT8), the model volume is compressed to 0.9MB, and the inference latency ≤ 10ms (ARM Cortex-M7), significantly reducing the computing power consumption compared with traditional Transformer models. The robustness is enhanced by adaptively matching the time-frequency distribution of signals with dynamic convolution kernels, and the anti-interference ability is significantly improved compared with SVM. The formula for the optimal convolution kernel size is as follows:

[0048]

[0049] (3) In terms of scalability, multi-modal expansion is supported. It can be extended to a 6-layer channel pyramid (up to 1024 channels). By dynamic parameter initialization (b init = μ band ·W b ), individual differences are eliminated, and the generalization performance is improved in scenarios such as fatigue detection and attention allocation. The migration ability in scenarios such as fatigue detection is verified. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The embodiments of the present application will be described in more detail by combining the accompanying drawings. The above and other objects, features, and advantages of the present application will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. They are used to explain the present application together with the embodiments of the present application, and do not constitute a limitation to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0051] Figure 1 FIG. is a schematic diagram of the system architecture of a classification method based on a hybrid neural dynamic encoder according to an embodiment of the present application.

[0052] Figure 2 FIG. is a schematic diagram of the EEG signal processing process of a driving behavior classification method based on a hybrid neural dynamic encoder according to an embodiment of the present application. Detailed implementation manners

[0053] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Instead, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0054] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0055] In the description of the embodiments of the present disclosure, the term "including" and its like terms should be understood as an open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects. There may also be other explicit and implicit definitions hereinafter.

[0056] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0057] Embodiment: Refer to Figure 1 、 Figure 2 , a driving behavior classification method based on a hybrid neural dynamic encoder, specifically including the following steps:

[0058] Step S1: EEG signal acquisition and preprocessing

[0059] 1.1 Signal acquisition

[0060] Collect the driver's EEG signals through a wireless EEG sensor (such as NeuroSky MindWave Mobile 2 or OpenBCI device), set the sampling rate to 256 Hz, and the electrode layout follows the international 10-20 system, focusing on covering the premotor cortex (F3, F4), parietal lobe (P3, P4) and movement-related areas (C3, C4). During the collection process, synchronously record the driving behavior event labels (such as braking, lane changing, accelerating, fatigue, distraction), and the time stamp accuracy is 1 ms.

[0061] 1.2 Wavelet denoising

[0062] The Morlet wavelet basis function is used because of its advantages in time-frequency localization characteristics and is suitable for the analysis of non-stationary EEG signals. The original signal is decomposed by wavelet for 6 layers, and the frequency band of 4 - 30 Hz (covering θ, α, and β rhythms) is retained, while the low-frequency baseline drift (<4 Hz) and high-frequency noise (>30 Hz) are removed.

[0063] The wavelet coefficients in the retained frequency band are processed with a hard threshold, and the threshold is set to 3σ, where σ is the standard deviation of the noise. The standard deviation of the noise is estimated by collecting EEG data during the period without signals, that is, when the driver is stationary and there is no obvious EEG activity. The purpose of the hard threshold processing is to set the wavelet coefficients smaller than the threshold to 0, thereby removing the interference of the noise and retaining the useful signal components.

[0064] Independent component analysis (ICA) separates the motion artifact components through the FastICA algorithm and calculates the artifact contribution degree. The formula is as follows:

[0065]

[0066] where i is the index variable used to traverse the artifact part, j is the index variable used to traverse all components, k is the number of artifact components, and n is the total number of components. When η > 0.3, signal re-sampling is triggered.

[0067] 1.4 Dynamic parameter initialization

[0068] Resting-state data acquisition: The driver closes his eyes and sits still for 2 minutes, and calculates the average energy μ of the θ (4 - 7 Hz), α (8 - 12 Hz), and β (13 - 30 Hz) frequency bands for each channel. band 。

[0069] Based on the driver's baseline EEG signal (resting state), calculate the individual frequency band energy mean μ band , and initialize the bias term b of the convolutional layer init = μ band ·W b , to eliminate the influence of individual physiological differences. Generate the preprocessed time-frequency feature matrix Μ ∈ R C×T , where C = 8 (number of key channels), and T = 128 (time window length of 0.5 seconds).

[0070] Step S2: Use hybrid neural dynamic coding

[0071] 2.1 Multi-scale frequency domain decomposition unit: Adopt a deformable depth convolutional kernel (Deformable DWConv) to dynamically adapt to the time-frequency distribution of the signal through the formula:

[0072]

[0073] Select the optimal kernel size \(K\in\{3,5,7\}\), where \(\varPsi(X)\) is the wavelet packet energy distribution, \(W\) k is the convolutional kernel weight matrix of size \(k\), and \(X\) is the time-frequency feature matrix obtained after preprocessing the input EEG signal. Every 100 training iterations, \(k\) is recalculated according to the formula adaptive .

[0074] Channel attention: Apply the SE Block to the output of each branch, and generate the channel weight \(W\) through global average pooling C .

[0075] 2.2 Hierarchical spatio-temporal attention mechanism

[0076] Spiking neural network temporal encoder: Pulse-code the features based on the LIF neuron model, and the membrane potential update formula is:[[]]

[0077]

[0078] where \(\tau\) is the decay coefficient, \(V\) th is the triggering threshold, \(\varTheta\) is the step function,[[]] represents the membrane potential of the neuron at time \(t\),[[]] represents the value of the \(j\)-th input signal.[[]]

[0079] 2.3 Spatio-temporal joint coding module:

[0080] On the spatial path, generate the brain region functional connection matrix by adopting the Graph Attention Network (GAT), and the node feature \(E\) node is the channel energy feature, and the time feature \(E\) time is the sliding window mean (window size 300ms).[[]]

[0081] The GAT layer contains 2 layers of attention heads, with 64 hidden units in each layer. The formula is as follows:[[]]

[0082] A dynamic = GAT(\(E\) node || \(E\) time )

[0083] On the time path, adopt the Squeeze-and-Excitation Temporal Convolution (SE-Conv), with the temporal convolution kernel size of \(5\times5\), dilation rate of 2, and padding of 2. Capture the evolution law of the behavior intention through the formula, and the formula is as follows:[[]]

[0084] F out = \(\sigma(W\) C \cdot GlobalAvgPool(F in ))\odot F in

[0085] where \(F\) inDenote the feature map input to the compressed excitation temporal convolutional module as W C is a learnable weight matrix.

[0086] Step S3: Driving behavior classification

[0087] 3.1 Classifier design

[0088] Fully connected network:

[0089] Input layer: 128-dimensional spatio-temporal features.

[0090] Hidden layer: 256-dimensional (ReLU activation) → 128-dimensional (ReLU activation).

[0091] Output layer: 5-dimensional (Softmax activation).

[0092] The model is trained using the cross-entropy loss function combined with L2 regularization. The cross-entropy loss function is used to measure the difference between the probability distribution predicted by the model and the true labels, and can effectively guide the model to train in the correct classification direction. The role of L2 regularization is to constrain the weight parameters of the model to prevent overfitting, and the regularization coefficient λ is set to 0.001.

[0093] 3.2 Softmax classifier

[0094] Input h st to the fully connected layer (dimension 256→128→5), and the activation function is ReLU.

[0095] Output the probability distribution of five types of driving behaviors: P = Softmax(W f h st +b f ), where h st is the spatio-temporal joint feature vector, and b f is a bias vector of length 5.

[0096] 3.3 Model optimization

[0097] The model parameters are updated using the AdamW optimizer. AdamW is an optimization algorithm that combines the Adam optimizer and L2 regularization. It can adaptively adjust the learning rate of each parameter while performing regularization on the weight parameters, which helps to improve the training efficiency and generalization ability of the model. The batch size is set to 64, that is, 64 samples are used for parameter update each time during training. The number of training epochs is set to 50, which means the model will perform 50 iterative trainings on the entire training dataset. An early stopping mechanism is introduced. During the training process, the validation set is used to evaluate the performance of the model. If the loss on the validation set does not decrease for 10 consecutive epochs, it is considered that the model has achieved good performance and the training is stopped to avoid overfitting.

[0098] 3.4 Model Quantization and Compression

[0099] Post-Training Quantization (PTQ): The model parameters are quantized from FP32 to INT8, using a symmetric quantization strategy, and the calibration dataset is 1000 randomly sampled EEG segments.

[0100] Performance Metrics: The model size is compressed to 0.9MB, the inference latency ≤ 10ms (based on the ARM Cortex-M7 microcontroller), and the classification accuracy degradation ≤ 1.5%.

Claims

1. A driving behavior classification method based on a hybrid neural dynamic encoder, characterized in that, Including the following steps: Step S1, EEG signal acquisition and preprocessing: The driver's EEG signal is collected by a wireless EEG sensor. After removing motion artifacts through wavelet denoising and independent component analysis (ICA), a preprocessed time-frequency feature matrix M ∈ R C×T is generated, where C is the number of EEG channels and T is the length of the time window. On this basis, the individual band energy mean μ is calculated based on the driver's resting-state EEG band , and through b init = μ band ·W b The bias of the convolutional layer is initialized to eliminate the influence of individual physiological differences on subsequent processing Step S2, using hybrid neural dynamic encoding: Extract the local frequency domain features of the EEG signal through a deformable depth convolution kernel, and model the spatial correlation between EEG channels and the temporal evolution law of driving behavior based on a hierarchical spatio-temporal attention mechanism, and output a spatio-temporal joint feature vector; Step S3, driving behavior classification: Input the spatio-temporal joint feature vector into a Softmax classifier to output the classification results of five types of driving behaviors.

2. The driving behavior classification method based on a hybrid neural dynamic encoder according to claim 1, characterized in that Step S1, EEG signal acquisition and preprocessing includes: Wavelet denoising uses Morlet wavelet basis functions to perform time-frequency decomposition on the EEG signal, and the reserved frequency band is 4 - 30 Hz; Step S1, EEG signal acquisition and preprocessing package uses independent component analysis to separate the motion artifact components through the Fast algorithm, and calculate the artifact contribution degree. The formula is as follows: Where i is the index variable for traversing the artifact part, j is the index variable for traversing all components, k is the number of artifact components, n is the total number of components, and when η > 0.3, signal resampling is triggered.

3. The driving behavior classification method based on a hybrid neural dynamic encoder according to claim 1, characterized in that The step S1, EEG signal acquisition and preprocessing includes: Calculate the mean value μ of the energy of each frequency band for the driver's baseline EEG signal (resting state). band , initialize the bias term of the convolutional layer to eliminate the influence of individual physiological differences: b init = μ band ·W b Among which W b is the weight coefficient.

4. The driving behavior classification method based on a hybrid neural dynamic encoder according to claim 1, wherein The step S2, using hybrid neural dynamic encoding specifically includes: Multi-scale frequency domain decomposition unit: Adopt a deformable depth convolution kernel (Deformable Depthwise Convolution, Deformable DWConv) to dynamically adapt to the signal time-frequency distribution through the formula: Select the optimal kernel size \(K\in\{3,5,7\}\), where \(\Psi(X)\) is the wavelet packet energy distribution, and \(W\) k is the convolutional kernel weight matrix of size \(k\), and \(X\) is the time-frequency feature matrix obtained after preprocessing the input EEG signal.

5. The driving behavior classification method based on the hybrid neural dynamic encoder according to claim 1, characterized in that, The step S2, using hybrid neural dynamic encoding specifically includes: Spiking neural network temporal encoder: Pulse-encode the features based on the LIF neuron model, and the membrane potential update formula is: where τ is the decay coefficient, V th is the trigger threshold, Θ is the step function, represents the membrane potential of the neuron at time t, represents the value of the j-th input signal.

6. The driving behavior classification method based on the hybrid neural dynamic encoder according to claim 1, wherein The step S2, using hybrid neural dynamic encoding specifically includes: Spatio-temporal joint encoding module: On the spatial path, generate a brain region functional connection matrix by using a graph convolutional network (Graph Convolutional Networks, GAT); A dynamic = GAT(E node ||E time ) E node Indicates the intervention of node features, E time Indicates the embedding of time features; On the temporal path, adopt a squeeze and excitation convolutional (SE-Conv) to capture the evolution law of behavior intention through the formula. The formula is as follows: F out = σ(W C · GlobalAvgPool(F in )) ⊙ F in Among them, F in represents the feature map input to the compressive excitation temporal convolutional module, and W C is a learnable weight matrix.

7. The driving behavior classification method based on the hybrid neural dynamic encoder according to claim 1, characterized in that, The step S2, using hybrid neural dynamic encoding specifically includes: The deformable depth convolution kernel extracts the energy features of different frequency bands through a multi-branch parallel structure, and dynamically adjusts the weights in combination with a channel attention module (Squeeze and Excitation, SE Block); The spiking neural network temporal encoder converts continuous features into discrete pulse sequences through a membrane potential accumulation and threshold triggering mechanism.

8. The driving behavior classification method based on the hybrid neural dynamic encoder according to claim 1, characterized in that For step S3, driving behavior classification, a lightweight fully connected network uses the ReLU activation function, and the output layer uses Softmax normalization.

9. An electronic device, characterized in that, Including a memory, a processor, and a computer program stored on the memory, when the processor executes the program, it implements the driving behavior classification method based on a hybrid neural dynamic encoder as described in any one of claims 1 - 8.

10. A computer-readable storage medium, characterized in that, Stored with a computer program, when the program is executed by the processor, it implements the driving behavior classification method based on a hybrid neural dynamic encoder as described in any one of claims 1 - 8.

Images