A driving behavior classification method based on a hybrid neural dynamic encoder

Abstract

The application discloses a driving behavior classification method based on a hybrid neural dynamic encoder and relates to the field of intelligent driving assistance and electroencephalogram signal classification. The main part is composed of electroencephalogram signal acquisition, hybrid neural dynamic coding and driving behavior classification. After the extracted EEG signal is subjected to wavelet denoising and independent component analysis to remove motion artifacts, a pretreated time-frequency feature matrix is generated; the neural dynamic coding module adopts a deformable deep convolution kernel to extract the local frequency domain features of the EEG signal and a hierarchical space-time attention mechanism to model the spatial correlation between EEG channels and the driving behavior time sequence evolution law, and outputs a space-time joint feature vector; the driving behavior classification module is responsible for realizing the real-time discrimination function of five types of driving behaviors. Through the fusion of biological neuron mechanism and dynamic coding, the application significantly improves the feature expression capability of the model on non-stationary electroencephalogram signals, and further improves the effect of electroencephalogram signal classification and recognition.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal classification, and more specifically, to a method for classifying driving behavior based on a hybrid neural dynamic encoder. Background Technology

[0002] Electroencephalography (EEG) signals are electrical wave changes formed by the summation of postsynaptic potentials generated synchronously by a large number of neurons during brain activity. They are a comprehensive reflection of the electrophysiological activity of brain nerve cells on the surface of the cerebral cortex or scalp. Electrodes are typically placed on the scalp to detect EEG signals, which are then collected and processed using relevant equipment.

[0003] With the development of driving behavior analysis technology, EEG-based driving behavior classification methods have been widely applied. However, existing classification methods still have some problems in practical use. For example, current classification methods on the market typically employ traditional signal processing and machine learning techniques, resulting in low classification accuracy and real-time performance, which are insufficient to meet the demands of modern intelligent driving systems for precise analysis and rapid response. This leads to problems such as low recognition accuracy and slow reaction speed in some scenarios. To improve performance, some researchers have attempted to improve classification results by introducing deep learning models. However, such improvements often face problems such as high model complexity, large computational load, and the need for a large amount of labeled data, limiting practical applications. A search revealed a visual Transformer-based EEG signal classification method with publication number CN114176607B, published on April 19, 2024. This method, through data preprocessing, feature extraction, and training of a Vision Transformer-based EEG signal classification model, achieves good performance in EEG signal classification tasks. However, this method is mainly suitable for static EEG signals; its classification accuracy and real-time performance are not ideal for dynamically changing driving behavior signals. Furthermore, this method has high requirements for data quality during feature extraction and model training, making it susceptible to noise and individual differences, resulting in insufficient model generalization ability. A search revealed a brain signal classification method based on brain functional connectivity network features, published on September 12, 2023, with publication number CN116541751B. This method collects motor imagery EEG signals, performs preprocessing and feature extraction, selects important EEG channels using brain functional connectivity networks, and then uses an SVM model for classification. While this method utilizes the features of brain functional connectivity networks and improves classification performance, it is primarily designed for specific motor imagery tasks. For complex and dynamically changing signals such as driving behavior, its adaptability and real-time performance are insufficient. Moreover, this method does not consider temporal features during channel selection and feature extraction, resulting in a weak ability to process time-varying signals. These problems indicate that current traditional classification methods on the market are ill-suited to effectively address the new demands for high accuracy and rapid response in complex driving behavior analysis.

[0004] Therefore, a driving behavior classification method based on a hybrid neural dynamic encoder is desired. Summary of the Invention

[0005] This invention provides a driving behavior classification method based on a hybrid neural dynamic encoder to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a driving behavior classification method based on a hybrid neural dynamic encoder, specifically including the following steps:

[0007] Step S1: EEG signal acquisition and preprocessing

[0008] 1.1 Signal Acquisition

[0009] Driver EEG signals were acquired using wireless EEG sensors (such as NeuroSky MindWave Mobile 2 or OpenBCI devices) at a sampling rate of 256Hz. Electrode placement followed the international 10-20 system, focusing on the premotor cortex (F3, F4), parietal lobe (P3, P4), and motor-related areas (C3, C4). Driving behavior event tags (such as braking, lane changing, acceleration, fatigue, and distraction) were recorded simultaneously during acquisition, with a timestamp accuracy of 1ms.

[0010] 1.2 Wavelet Denoising

[0011] Morlet wavelet basis functions are suitable for non-stationary EEG signal analysis due to their advantages in time-frequency localization. The original signal undergoes a 6-level wavelet decomposition, retaining the 4-30Hz frequency band (covering θ, α, and β rhythms) while removing low-frequency baseline drift (<4Hz) and high-frequency noise (>30Hz). The wavelet coefficients of the retained frequency band are thresholded (hard threshold, threshold value is 3σ, where σ is the noise standard deviation). The reconstructed signal is denoted as...

[0012] 1.3 Independent Component Analysis

[0013] Independent Component Analysis (ICA) separates motion artifacts using the FastICA algorithm and calculates their contribution, as shown in the following formula:

[0014]

[0015] Where i is the index variable used to traverse the artifact part, j is the index variable used to traverse all components, k is the number of artifact components, n is the total components, and signal resampling is triggered when η>0.3.

[0016] 1.4 Dynamic Parameter Initialization

[0017] The mean energy μ of the individual frequency band was calculated based on the driver's baseline EEG signal (resting state). band Initialize the convolutional layer bias term b init =μ band ·W b To eliminate the influence of individual physiological differences, a preprocessed time-frequency feature matrix M∈R is generated. C×T Where C = 8 (number of critical channels), T = 128 (time window length 0.5 seconds).

[0018] Step S2: Utilizing hybrid neural dynamic coding

[0019] 2.1 Multi-scale frequency domain decomposition unit: A deformable deep convolution kernel (DWConv) is used to dynamically adapt the time-frequency distribution of the signal, using the formula:

[0020]

[0021] Choose the optimal kernel size K∈{3,5,7}, where Ψ(X) is the wavelet packet energy distribution, and W k Let X be the convolution kernel weight matrix of size k, and let X be the time-frequency feature matrix obtained after preprocessing the input EEG signal.

[0022] 2.2 Hierarchical Spatiotemporal Attention Mechanism

[0023] Spiking Neural Network Temporal Encoder: Based on the LIF neuron model, features are pulse-encoded, and the membrane potential update formula is:

[0024]

[0025] Where τ is the attenuation coefficient, V th The threshold value is Θ, and Θ is the step function. This represents the membrane potential of the neuron at time t. This represents the value of the j-th input signal.

[0026] 2.3 Spatiotemporal Joint Coding Module:

[0027] In the spatial pathway, a brain region functional connectivity matrix is ​​generated using a dynamic graph convolutional network (GAT):

[0028] A dynamic =GAT(E node ||E time )

[0029] E node Indicates node feature intervention, E time Represents temporal feature embedding;

[0030] In the temporal path, compressed excitation temporal convolution (SE-Conv) is used to capture the evolutionary pattern of behavioral intent through a formula, as follows:

[0031] F out =σ(W C ·GlobalAvgPool(F in ))⊙F in

[0032] Among them, F in W represents the feature map input to the compressed-excitation temporal convolution module. CIt is a learnable weight matrix.

[0033] Step S3: Classification of Driving Behavior

[0034] 3.1 Softmax Classifier

[0035] Enter h st The fully connected layer (dimensions 256→128→5) uses ReLU as the activation function.

[0036] Output the probability distribution of the five types of driving behaviors: P = Softmax(W f h st +b f )in h st b is the spatiotemporal joint eigenvector. f It is a bias vector of length 5.

[0037] 3.2 Model Quantization and Compression

[0038] Post-training quantization (PTQ): The model parameters are quantized from FP32 to INT8 using a symmetric quantization strategy. The calibration dataset consists of 1000 randomly sampled EEG fragments.

[0039] Performance metrics: Model size compressed to 0.9MB, inference latency ≤10ms (based on ARM Cortex-M7 microcontroller), classification accuracy maintained at ≤1.5% decrease.

[0040] 3.3 Real-time Deployment Optimization

[0041] Memory allocation: A static memory pre-allocation strategy is adopted to avoid dynamic memory fragmentation.

[0042] Multi-threaded pipeline: Signal acquisition, preprocessing, and inference are executed in separate threads, with a frame processing cycle of ≤50ms.

[0043] An electronic device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the program to implement the driving behavior classification method based on a hybrid neural dynamic encoder.

[0044] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the driving behavior classification method based on a hybrid neural dynamic encoder.

[0045] Compared with the prior art, the advantages of the present invention are as follows:

[0046] (1) In terms of methodological innovation, this invention is the first to combine a LIF spiking neural network with a deformable deep convolutional kernel, achieving brain-like spiking encoding of features through dynamic membrane potential updates, reducing redundant information by 80% compared to traditional continuous value features. A dual-path attention mechanism (dynamic graph convolution + compressed excitation temporal convolution) is constructed to dynamically capture functional connectivity of brain regions (A... dynamic =GAT(E node ||E time Time sequence evolution law (F) out =σ(W C ·GlobalAvgPool(F in ))⊙F in This method improves the spatiotemporal feature representation capability compared to the traditional CNN-LSTM.

[0047] (2) In terms of performance, the number of model parameters can be further compressed. Through pulse coding and model quantization (INT8), the model size is compressed to 0.9MB, and the inference latency is ≤10ms (ARM Cortex-M7), significantly reducing computational power consumption compared to the traditional Transformer model. Robustness is enhanced by adaptively matching the time-frequency distribution of the signal using dynamic convolution kernels, significantly improving anti-interference capability compared to SVM. The optimal convolution kernel size formula is as follows:

[0048]

[0049] (3) In terms of scalability, it supports multimodal expansion. It can be expanded to a 6-layer channel pyramid (up to 1024 channels), and can be initialized through dynamic parameters (b init =μ band ·W b Eliminating individual differences improves generalization performance in scenarios such as fatigue detection and attention allocation. Validating transferability in scenarios such as fatigue detection. Attached Figure Description

[0050] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0051] Figure 1 This is a schematic diagram of the system architecture of the classification method based on the hybrid neural dynamic encoder according to an embodiment of this application.

[0052] Figure 2 This is a schematic diagram of the EEG signal processing of the driving behavior classification method based on a hybrid neural dynamic encoder according to an embodiment of this application. Detailed Implementation

[0053] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0054] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0055] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0056] It should be noted that the use of the terms "a" or "a plurality of" in this disclosure is illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0057] Example: See Figure 1 , Figure 2 A driving behavior classification method based on a hybrid neural dynamic encoder specifically includes the following steps:

[0058] Step S1: EEG signal acquisition and preprocessing

[0059] 1.1 Signal Acquisition

[0060] Driver EEG signals were acquired using wireless EEG sensors (such as NeuroSky MindWave Mobile 2 or OpenBCI devices) at a sampling rate of 256Hz. Electrode placement followed the international 10-20 system, focusing on the premotor cortex (F3, F4), parietal lobe (P3, P4), and motor-related areas (C3, C4). Driving behavior event tags (such as braking, lane changing, acceleration, fatigue, and distraction) were recorded simultaneously during acquisition, with a timestamp accuracy of 1ms.

[0061] 1.2 Wavelet Denoising

[0062] Morlet wavelet basis functions are suitable for non-stationary EEG signal analysis due to their advantages in time-frequency localization characteristics. The original signal is decomposed into 6 levels of wavelet decomposition, retaining the 4-30Hz frequency band (covering θ, α, and β rhythms) and removing low-frequency baseline drift (<4Hz) and high-frequency noise (>30Hz).

[0063] Hard thresholding was applied to the wavelet coefficients in the retained frequency band, with a threshold set to 3σ, where σ is the noise standard deviation. The noise standard deviation was estimated by collecting EEG data during periods of no signal, i.e., when the driver was stationary and showed no significant EEG activity. The purpose of hard thresholding was to set wavelet coefficients smaller than the threshold to 0, thereby removing noise interference and retaining useful signal components.

[0064] Independent Component Analysis (ICA) separates motion artifacts using the FastICA algorithm and calculates their contribution, as shown in the following formula:

[0065]

[0066] Where i is the index variable used to traverse the artifact part, j is the index variable used to traverse all components, k is the number of artifact components, n is the total components, and signal resampling is triggered when η>0.3.

[0067] 1.4 Dynamic Parameter Initialization

[0068] Resting-state data acquisition: The driver sat still with eyes closed for 2 minutes, and the average energy μ of each channel in the θ (4-7Hz), α (8-12Hz), and β (13-30Hz) frequency bands was calculated. band .

[0069] The mean energy μ of the individual frequency band was calculated based on the driver's baseline EEG signal (resting state). band Initialize the convolutional layer bias term b init =μ band ·W b To eliminate the influence of individual physiological differences, a preprocessed time-frequency feature matrix M∈R is generated. C×T Where C = 8 (number of critical channels), T = 128 (time window length 0.5 seconds).

[0070] Step S2: Utilizing hybrid neural dynamic coding

[0071] 2.1 Multi-scale frequency domain decomposition unit: A deformable deep convolution kernel (DWConv) is used to dynamically adapt the time-frequency distribution of the signal, using the formula:

[0072]

[0073] Choose the optimal kernel size K∈{3,5,7}, where Ψ(X) is the wavelet packet energy distribution, and W k Let be the convolutional kernel weight matrix of size k, and X be the time-frequency feature matrix obtained after preprocessing the input EEG signal. Every 100 training iterations, k is recalculated according to the formula. adaptive .

[0074] Channel attention: Apply the SE Block to the output of each branch, and generate channel weights W through global average pooling. C .

[0075] 2.2 Hierarchical Spatiotemporal Attention Mechanism

[0076] Spiking Neural Network Temporal Encoder: Based on the LIF neuron model, features are pulse-encoded, and the membrane potential update formula is:

[0077]

[0078] Where τ is the attenuation coefficient, V th The threshold value is Θ, and Θ is the step function. This represents the membrane potential of the neuron at time t. This represents the value of the j-th input signal.

[0079] 2.3 Spatiotemporal Joint Coding Module:

[0080] In the spatial pathway, a brain region functional connectivity matrix is ​​generated using a dynamic graph convolutional network (GAT), and node features E node Channel energy characteristics, time characteristics E time This is the average value of the sliding window (window size 300ms).

[0081] The GAT layer contains two attention heads, each with 64 hidden units. The formula is as follows:

[0082] A dynamic =GAT(E node ||E time )

[0083] In the temporal path, compressed excitation temporal convolution (SE-Conv) is used, with a kernel size of 5×5, an expansion ratio of 2, and padding of 2. The evolutionary pattern of behavioral intent is captured using the following formula:

[0084] F out =σ(W C ·GlobalAvgPool(F in ))⊙F in

[0085] Among them, F inW represents the feature map input to the compressed-excitation temporal convolution module. C It is a learnable weight matrix.

[0086] Step S3: Classification of Driving Behavior

[0087] 3.1 Classifier Design

[0088] Fully connected networks:

[0089] Input layer: 128-dimensional spatiotemporal features.

[0090] Hidden layer: 256 dimensions (ReLU activation) → 128 dimensions (ReLU activation).

[0091] Output layer: 5-dimensional (Softmax activation).

[0092] The model is trained using a combination of cross-entropy loss and L2 regularization. Cross-entropy loss measures the difference between the model's predicted probability distribution and the true label, effectively guiding the model towards the correct classification. L2 regularization constrains the model's weight parameters to prevent overfitting; the regularization coefficient λ is set to 0.001.

[0093] 3.2 Softmax Classifier

[0094] Enter h st The fully connected layer (dimensions 256→128→5) uses ReLU as the activation function.

[0095] Output the probability distribution of five types of driving behavior: P = Softmax(W f h st +b f ),in h st b is the spatiotemporal joint eigenvector. f It is a bias vector of length 5.

[0096] 3.3 Model Optimization

[0097] The AdamW optimizer is used to update model parameters. AdamW is an optimization algorithm that combines the Adam optimizer and L2 regularization. It adaptively adjusts the learning rate of each parameter while regularizing the weight parameters, which helps improve the training efficiency and generalization ability of the model. The batch size is set to 64, meaning that 64 samples are used for parameter updates during each training iteration. The number of training epochs is set to 50, meaning that the model will be trained on the entire training dataset for 50 iterations. An early stopping mechanism is introduced. During training, the validation set is used to evaluate the model's performance. If the loss on the validation set does not decrease for 10 consecutive epochs, the model is considered to have reached good performance, and training is stopped to avoid overfitting.

[0098] 3.4 Model Quantization and Compression

[0099] Post-training quantization (PTQ): The model parameters are quantized from FP32 to INT8 using a symmetric quantization strategy. The calibration dataset consists of 1000 randomly sampled EEG fragments.

[0100] Performance metrics: Model size compressed to 0.9MB, inference latency ≤10ms (based on ARM Cortex-M7 microcontroller), classification accuracy maintained at ≤1.5% decrease.

Claims

1. A driving behavior classification method based on a hybrid neural dynamic encoder, characterized in that, Includes the following steps: Step S1: EEG signal acquisition and preprocessing: The driver's EEG signal is acquired using a wireless EEG sensor. After wavelet denoising and independent component analysis (ICA) to remove motion artifacts, a preprocessed time-frequency feature matrix is ​​generated. Where C is the number of EEG channels and T is the time window length; based on this, the average energy of each individual frequency band is calculated based on the driver's resting-state EEG. and through Complete the initialization of convolutional layer bias terms. These are weighting coefficients, used to eliminate the influence of individual physiological differences on subsequent treatments. Step S2: Utilize hybrid neural dynamic coding: Extract local frequency domain features of EEG signals through variable deep convolution kernels, and model the spatial correlation between EEG channels and the temporal evolution of driving behavior based on a hierarchical spatiotemporal attention mechanism to output a spatiotemporal joint feature vector. Step S3, Driving Behavior Classification: Input the spatiotemporal joint feature vector into the Softmax classifier and output five categories of driving behavior classification results; Step S2, which utilizes hybrid neural dynamic coding, specifically includes: Multi-scale frequency domain decomposition unit: Employs deformable depthwise convolution (Deformable DWConv) kernels to dynamically adapt the signal's time-frequency distribution, using the formula: Select the optimal core size ,in, For wavelet packet energy distribution, Let X be the convolution kernel weight matrix of size k, and let X be the time-frequency feature matrix obtained after preprocessing the input EEG signal. The optimal deformable depthwise convolution kernel size is selected adaptively; k is the convolution kernel size traversal variable. For Frobenius norm, subscript This serves as the standard identifier for the norm. Step S2, utilizing hybrid neural dynamic coding, specifically includes: Spiking Neural Network Temporal Encoder: Based on the LIF neuron model, features are pulse-encoded, and the membrane potential update formula is: Where τ is the attenuation coefficient, The threshold value is Θ, and Θ is the step function. This represents the membrane potential of the neuron at time t. This represents the value of the j-th input signal; Step S2, utilizing hybrid neural dynamic coding, specifically includes: Spatiotemporal co-coding module: In the spatial pathway, brain region functional connectivity matrices are generated using Graph Convolutional Networks (GAT): This indicates the involvement of node features. Represents temporal feature embedding; It is the core output of the spatial pathway, a dynamic brain region functional connectivity matrix, used to characterize the dynamic co-activation relationship of different brain regions during driving; In the temporal pathway, squeezed and excitation convolutional (SE-Conv) is employed to capture the evolutionary patterns of behavioral intent using the following formula: in, This represents the feature map input to the compressed-excitation temporal convolution module. For a learnable weight matrix, It is the global average pooling operator, a universal standard operator in the field of deep learning. It is the temporal characteristics of the driver's EEG after pulse coding is completed.

2. The driving behavior classification method based on a hybrid neural dynamic encoder according to claim 1, characterized in that, Step S1, EEG signal acquisition and preprocessing, includes: wavelet denoising, which uses Morlet wavelet basis functions to perform time-frequency decomposition on the EEG signal, retaining the frequency band of 4-30Hz; Step S1, the EEG signal acquisition and preprocessing package, uses independent component analysis (ICA) with the Fast algorithm to separate motion artifact components and calculate the artifact contribution, as shown in the following formula: Where i is the index variable used to traverse the artifact portion, j is the index variable used to traverse all components, k is the number of artifact components, and n is the total number of components. Signal resampling is triggered when η > 0.

3. The i-th motion artifact independent component separated by FastICA corresponds to the EEG voltage time series of the specified time length. This is the j-th complete independent component isolated by FastICA.

3. The driving behavior classification method based on a hybrid neural dynamic encoder according to claim 1, characterized in that, Step S1, EEG signal acquisition and preprocessing, includes: Individual frequency band energy mean calculated based on the driver's baseline resting-state EEG signal. Initialize the convolutional layer bias terms to eliminate the influence of individual physiological differences: in These are the weighting coefficients.

4. The driving behavior classification method based on a hybrid neural dynamic encoder according to claim 1, characterized in that, Step S2, utilizing hybrid neural dynamic coding, specifically includes: Deformable deep convolutional kernels extract energy features of different frequency bands through a multi-branch parallel structure and dynamically adjust weights by combining a channel attention module (Squeeze and Excitation, SE Block); the spiking neural network temporal encoder converts continuous features into discrete pulse sequences through membrane potential accumulation and threshold triggering mechanisms.

5. The driving behavior classification method based on a hybrid neural dynamic encoder according to claim 1, characterized in that, Step S3, driving behavior classification, uses a lightweight fully connected network with ReLU activation function, and the output layer uses Softmax normalization.

6. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the program to implement the driving behavior classification method based on a hybrid neural dynamic encoder as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The system contains a computer program that, when executed by a processor, implements the driving behavior classification method based on a hybrid neural dynamic encoder as described in any one of claims 1-5.

Images