A driving intention recognition method based on functional connection and graph neural network

Abstract

The application provides a driving intention recognition method based on functional connectivity and a graph neural network, comprising the following steps: S1, collecting electroencephalogram signals of a driver during driving, and preprocessing original electroencephalogram data; S2, calculating power spectral densities of each frequency band of the preprocessed electroencephalogram signals as electroencephalogram signal frequency domain features; S3, constructing an adjacency matrix as an initial graph structure based on electrode spatial proximity and functional connectivity; and S4, inputting the frequency domain features in S2 and the adjacency matrix obtained in S3 into a graph attention network for feature aggregation, inputting the extracted feature expression into a classifier to classify driving intentions and outputting results. The method can solve the problems of single electroencephalogram feature extraction and poor expression ability of the classification model in the existing driving intention prediction method, improve the accuracy of driving intention classification, improve the interpretability of the classification results, and better apply the method to driving state perception and auxiliary decision-making of a human-machine co-driving system.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving and EEG signal processing technology, specifically to a driving intention recognition method based on functional connectivity and graph neural networks. Background Technology

[0002] The emergence of artificial intelligence has led to the rapid development of autonomous driving technology. However, due to the complexity of road environments, achieving fully autonomous driving without human intervention still faces many challenges. For the foreseeable future, simultaneous human-machine control of vehicles will persist. Human-machine collaboration technology thus becomes a crucial support for this transitional phase. This involves the autonomous driving system interacting with the environment and with other vehicles through sensors and other devices, while simultaneously acquiring the driver's perception and decision-making information. This further facilitates interaction between the driver and the system, enabling collaborative operation. Driving intent, as a vital piece of decision-making information, provides crucial support for autonomous driving systems, and the accuracy of driving intent recognition is one of the key issues determining human-machine collaboration and autonomous driving.

[0003] Electroencephalogram (EEG) signals contain high-frequency neural activity information, which can fully reflect the driver's physiological and psychological state. Extracting key information from EEG can not only enable the perception of the driver's state, but also enable the classification and recognition of driving intentions by building a neural network classifier, providing a new approach for autonomous driving systems to make decisions and perceive.

[0004] There are natural spatial topological structures and functional connections between EEG electrodes. Driving behavior also involves multiple areas of the brain, and the interactions and functional connections between EEG signals during driving are crucial in EEG-based driving behavior analysis. Existing methods for identifying driving intentions often classify and detect them from the perspectives of frequency domain analysis, time domain analysis, time-frequency analysis, and nonlinear analysis. However, the aforementioned spatial features of EEG signals and the functional connections between electrodes are often overlooked, leading to the loss of some information and affecting the accuracy of the final detection.

[0005] Therefore, there is an urgent need for recognition methods that can comprehensively extract multiple features from the frequency domain, spatial domain, and functional connectivity to achieve the maximum benefit of driving intent classification and detection. Summary of the Invention

[0006] The present invention aims to provide a driving intention recognition method based on functional connectivity and graph neural networks to solve the above problems.

[0007] The technical solution of this invention is: a driving intention recognition method based on functional connectivity and graph neural networks, comprising:

[0008] Step S1: Collect the driver's EEG signals while driving and preprocess the raw EEG data;

[0009] Step S2: Calculate the power spectral density of each frequency band of the preprocessed EEG signal as the frequency domain feature of the EEG signal.

[0010] Step S3: Construct an adjacency matrix as the initial graph structure based on the spatial proximity and functional connectivity of the electrodes;

[0011] Step S4: The frequency domain features from step S2 and the adjacency matrix obtained in step S3 are input to the attention network for feature aggregation. The extracted feature representation is then input to the classifier to classify the driving intention and output the result.

[0012] Preferably, step S1 specifically includes:

[0013] Step S11: Collect EEG data of the driver under two types of driving behaviors: constant speed driving and braking. The collected data is processed by bandpass filtering from 0.5HZ to 70HZ.

[0014] Step S12: Remove the artifact components of electrooculography using independent component analysis.

[0015] Step S13: Remove damaged electrodes and supplement them by interpolation with adjacent electrodes;

[0016] Step S14: The raw EEG data is divided into non-overlapping time windows of 1 second each, based on the two types of driving intention labels: constant speed driving and braking, and then divided into training set and test set.

[0017] Preferably, step S2 specifically includes:

[0018] Step S21: The EEG signal is decomposed into five bands by fast Fourier transform, namely delta band (0.5-3Hz), theta band (4-7Hz), alpha band (8-13Hz), beta band (14-30Hz) and gamma band (31-70Hz).

[0019] Step S22: Calculate the power spectral density of the five bands in S21 as the frequency domain characteristics of the EEG signal. The formula for calculating the power spectral density is:

[0020]

[0021] In the formula, S(w) represents the power spectral density in the corresponding frequency band; N represents the number of samplings; Δt represents the sampling period; f s X(w) represents the sampling frequency; X(w) represents the EEG signals of each band in S21.

[0022] Preferably, step S3 specifically includes:

[0023] Step S31: Treat the electrode channels as nodes in a spatial structure graph, connect each pair of nodes as edges of the network, use Euclidean distance as the weight value on each edge, and construct a spatial connection matrix A0(i,j) by setting a threshold:

[0024]

[0025] In the formula, A0(i, h) represents the spatial connectivity matrix, and D ij This represents the Euclidean distance between any two points, and Threshold represents the set distance threshold, which is the average of all distances between electrodes.

[0026] Step S32: The correlation between each node is calculated using the phase lag index as the weight of each edge to construct the PLI matrix; wherein, the PLI matrix is ​​the phase lag index matrix, which represents the functional connectivity between electrodes by calculating the phase difference between the EEG signals of each electrode.

[0027]

[0028] In the formula, Δφ rel (t n ) represents t n The phase difference of the electrode signal at time point N represents the number of time points, and sign(number) is the sign function. If number is greater than 0, sign returns 1; if number is equal to 0, sign returns 0; if number is less than 0, sign returns -1.

[0029] Step S33: Binarize the PLI matrix obtained in S32 by setting a threshold to obtain the functional connectivity matrix A1(i,j); The binarization process is as follows: In the PLI matrix, elements greater than the threshold are set to 1, indicating that there is a functional connectivity relationship between the two electrode EEG signals; elements less than the threshold are set to 0, indicating that there is no functional connectivity relationship between the two electrode EEG signals.

[0030] Step S34: Integrate the functional connectivity and spatial proximity of different brain regions to construct the final adjacency matrix A(i,j), specifically:

[0031]

[0032] In the formula, A(i,j) represents the final adjacency matrix, A0(i,j) represents the spatial connectivity matrix, and A1(i,j) represents the functional adjacency matrix.

[0033] Preferably, step S4 specifically includes:

[0034] Step S41: Input the frequency domain features and graph structure into the graph attention network for feature aggregation;

[0035] Step S42: The feature map obtained after feature aggregation is flattened and read out through a fully connected layer. The read feature representation Z is input into a softmax layer to obtain the probabilities of the two driving intention classification tasks: braking and following. The output result is the task represented by the label encoding with the highest probability value. The softmax layer is specifically as follows:

[0036]

[0037] In the formula, P k Z represents the probability value of the k-th task; Z represents the feature representation of the input; C represents the number of driving intention classification categories.

[0038] Step S43: Select cross-entropy loss as the loss function, and use the Adam algorithm for gradient descent to optimize the model; the cross-entropy formula is as follows:

[0039]

[0040] In the formula, C represents the number of driving intent categories; c represents the category number; p c The probability distribution [p1, p2] of the braking and following vehicle intention labels output by the classification model; y represents the true category label after one-hot encoding, i.e., the C-dimensional vector after one-hot encoding. c This represents the true category label for class c.

[0041] Preferably, in step S43, y c The specific calculation is as follows:

[0042] A classification experiment was conducted with two driving intentions: the first category being following another vehicle and the second being braking (C=2). The true class label for the samples was braking, and the label y was obtained after one-hot encoding. c It is [0,1]; the true class label of the sample is "following the car", and the label y is obtained after one-hot encoding. c It is [1,0].

[0043] Preferably, step S41 specifically includes:

[0044] Step S411: Map the features of nodes i and j to vector W respectively, concatenate the results, map the concatenated vector to real numbers using a feedforward neural network a, activate it with the LeakyRELU function, and obtain the importance coefficient e of node j to node i. ij :

[0045]

[0046] in, N represents the feature vectors of nodes i and j; i represents all neighboring nodes of node i; W and a represent trainable parameters in the model.

[0047] Step S412: Update the importance coefficients using the adjacency matrix A(i, j). That is, when an element in the adjacency matrix is ​​greater than 0, it indicates that two nodes are connected, and the importance coefficient at that position is retained; otherwise, it is set to negative infinity. The updated importance coefficients are then normalized using softmax to obtain the relationship between node i and its neighboring nodes j ∈ N. i Attention coefficient a ij :

[0048]

[0049] Where k represents the neighboring nodes of node i, and N i Represents all neighboring nodes of node i;

[0050] Step S413: After obtaining the attention coefficients, feature aggregation is performed on all neighboring nodes j of node i using a weighted average method.

[0051]

[0052] Where, h′ i h represents the feature vector of node i after aggregation; j The features of neighbor node j are represented by σ; σ represents the vector aggregation function, which is a weighted average here; a ij represents the attention coefficient; W is a trainable parameter.

[0053] The beneficial effects of this invention are: solving the problem that existing driving intention prediction methods extract only single EEG features and have poor classification model expressive ability, improving the accuracy of driving intention classification while improving the interpretability of classification results, and better applying it to driving state perception and auxiliary decision-making in human-machine collaborative driving systems. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the overall process of a driving intention recognition method based on functional connectivity and graph neural networks, as provided in an embodiment of the present invention.

[0055] Figure 2 This is a structural diagram of the classification model in a driving intention recognition method based on functional connectivity and graph neural networks provided in an embodiment of the present invention. Detailed Implementation

[0056] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. The embodiments of the present invention are not limited thereto.

[0057] Example 1

[0058] like Figure 1 As shown, a driving intention recognition method based on functional connectivity and graph neural networks includes:

[0059] Step S1: Collect the driver's EEG signals while driving and preprocess the raw EEG data;

[0060] Step S2: Calculate the power spectral density of each frequency band of the preprocessed EEG signal as the frequency domain feature of the EEG signal.

[0061] Step S3: Construct an adjacency matrix as the initial graph structure based on the spatial proximity and functional connectivity of the electrodes;

[0062] Step S4: The frequency domain features from step S2 and the adjacency matrix obtained in step S3 are input to the attention network for feature aggregation. The extracted feature representation is then input to the classifier to classify the driving intention and output the result.

[0063] Step S1 specifically includes:

[0064] Step S11: Collect EEG data of the driver under two types of driving behaviors: constant speed driving and braking. The collected data is processed by bandpass filtering from 0.5HZ to 70HZ.

[0065] Step S12: Remove the artifact components of electrooculography using independent component analysis.

[0066] Step S13: Remove damaged electrodes and supplement them by interpolation with adjacent electrodes;

[0067] Step S14: The raw EEG data is divided into non-overlapping time windows of 1 second each, based on the two types of driving intention labels: constant speed driving and braking, and then divided into training set and test set.

[0068] Step S2 specifically includes:

[0069] Step S21: The EEG signal is decomposed into five bands by fast Fourier transform, namely delta band (0.5-3Hz), theta band (4-7Hz), alpha band (8-13Hz), beta band (14-30Hz) and gamma band (31-70Hz).

[0070] Step S22: Calculate the power spectral density of the five bands in S21 as the frequency domain characteristics of the EEG signal. The formula for calculating the power spectral density is:

[0071]

[0072] In the formula, S(w) represents the power spectral density in the corresponding frequency band; N represents the number of samplings; Δt represents the sampling period; f s X(w) represents the sampling frequency; X(w) represents the EEG signals of each band in S21.

[0073] Step S3 specifically includes:

[0074] Step S31: Treat the electrode channels as nodes in a spatial structure graph, connect each pair of nodes as edges of the network, use Euclidean distance as the weight value on each edge, and construct a spatial connection matrix A0(i,j) by setting a threshold:

[0075]

[0076] In the formula, A0(i,j) represents the spatial connectivity matrix, and D ij This represents the Euclidean distance between any two points, and Threshold represents the set distance threshold, which is the average of all distances between electrodes.

[0077] Step S32: The correlation between each node is calculated using the phase lag index as the weight of each edge to construct the PLI matrix; wherein, the PLI matrix is ​​the phase lag index matrix, which represents the functional connectivity between electrodes by calculating the phase difference between the EEG signals of each electrode.

[0078]

[0079] In the formula, Δφ rel (tn) represents t n The phase difference of the electrode signal at time point N represents the number of time points, and sign(number) is the sign function. If number is greater than 0, sign returns 1; if number is equal to 0, sign returns 0; if number is less than 0, sign returns -1.

[0080] Step S33: Binarize the PLI matrix obtained in S32 by setting a threshold to obtain the functional connectivity matrix A1(i,j); The binarization process is as follows: In the PLI matrix, elements greater than the threshold are set to 1, indicating that there is a functional connectivity relationship between the two electrode EEG signals; elements less than the threshold are set to 0, indicating that there is no functional connectivity relationship between the two electrode EEG signals.

[0081] Step S34: Integrate the functional connectivity and spatial proximity of different brain regions to construct the final adjacency matrix A(i,j), specifically:

[0082]

[0083] In the formula, A(i,j) represents the final adjacency matrix, A0(i,j) represents the spatial connectivity matrix, and A1(i,j) represents the functional adjacency matrix.

[0084] Step S4 specifically includes:

[0085] Step S41: Input the frequency domain features and graph structure into the graph attention network for feature aggregation;

[0086] Step S42: The feature map obtained after feature aggregation is flattened and read out through a fully connected layer. The read feature representation Z is input into a softmax layer to obtain the probabilities of the two driving intention classification tasks: braking and following. The output result is the task represented by the label encoding with the highest probability value. The softmax layer is specifically as follows:

[0087]

[0088] In the formula, P k Z represents the probability value of the k-th task; Z represents the feature representation of the input; C represents the number of driving intention classification categories.

[0089] The meanings of Zk and Zc are related to y in the following text. c The meaning is similar; if the total number of categories, i.e., C, is 2, then the feature representation Z read from the fully connected layer in S42 is a 2-dimensional vector [a, b]. In the classification experiment of two driving intentions, the first class (k=1) is following the car and the second class is braking, then Zk (when k=1) represents the feature representation a of the following category; then Zk (when k=2) represents the feature representation b of the braking category. The probability of following the car can be obtained by exp(Zk(k=1)) / (exp(Zk(k=1)+Zk(k=2)) (or simply expressed as exp(a) / (exp(a)+exp(b)). Here, exp(Zk(k=1)+Zk(k=2)) is... It represents the sum of all analogous features.

[0090] Finally, after softmax output, we obtain Pk = [probability of following the car, probability of braking].

[0091] Step S43: Select cross-entropy loss as the loss function, and use the Adam algorithm for gradient descent to optimize the model; the cross-entropy formula is as follows:

[0092]

[0093] In the formula, C represents the number of driving intent categories; c represents the category number; p c The probability distribution [p1, p2] of the braking and following vehicle intention labels output by the classification model; y represents the true category label after one-hot encoding, i.e., the C-dimensional vector after one-hot encoding.c This represents the true category label for class c.

[0094] In this loss function, the loss value is calculated from two parts: the predicted value of the driving intention and the true value label. The predicted value refers to the probability distribution of the driving intention category predicted by the proposed model: [probability of following another vehicle, probability of braking]; the true value refers to the actual driving behavior at each time period in the driving experiment. For example, in an experiment where the first category is following another vehicle and the second category is braking, if the actual situation is following another vehicle, the label obtained after one-hot encoding is [1,0]; if the actual situation is braking, the label obtained after one-hot encoding is [0,1]. (This has the same meaning as the probability distribution in the predicted value above. Since it is the true value, the 'probability' can only be either 0 or 1.)

[0095] In step S43, y c The specific calculation is as follows:

[0096] A classification experiment was conducted with two driving intentions: the first category being following another vehicle and the second being braking (C=2). The true class label for the samples was braking, and the label y was obtained after one-hot encoding. c It is [0,1]; the true class label of the sample is "following the car", and the label y is obtained after one-hot encoding. c It is [1,0].

[0097] like Figure 2 As shown, step S41 specifically includes:

[0098] Step S411: Map the features of nodes i and j to vector W respectively, concatenate the results, map the concatenated vector to real numbers using a feedforward neural network a, activate it with the LeakyRELU function, and obtain the importance coefficient e of node j to node i. ij :

[0099]

[0100] in, N represents the feature vectors of nodes i and j; i represents all neighboring nodes of node i; W and a represent trainable parameters in the model.

[0101] Step S412: Update the importance coefficients using the adjacency matrix A(i, j). That is, when an element in the adjacency matrix is ​​greater than 0, it indicates that two nodes are connected, and the importance coefficient at that position is retained; otherwise, it is set to negative infinity. The updated importance coefficients are then normalized using softmax to obtain the relationship between node i and its neighboring nodes j ∈ N. i Attention coefficient a ij :

[0102]

[0103] Where k represents the neighboring nodes of node i, and N i Represents all neighboring nodes of node i;

[0104] Step S413: After obtaining the attention coefficients, feature aggregation is performed on all neighboring nodes j of node i using a weighted average method.

[0105]

[0106] Where, h′ i h represents the feature vector of node i after aggregation; j The features of neighbor node j are represented by σ; σ represents the vector aggregation function, which is a weighted average here; a ij represents the attention coefficient; W is a trainable parameter.

[0107] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of one embodiment, and the processes depicted in the drawings are not necessarily essential for implementing the present invention.

Claims

1. A method for recognizing driving intentions based on functional connectivity and graph neural networks, characterized in that, include: Step S1: Collect the driver's EEG signals while driving and preprocess the raw EEG data; Step S2: Calculate the power spectral density of each frequency band of the preprocessed EEG signal as the frequency domain feature of the EEG signal. Step S3: Construct an adjacency matrix as the initial graph structure based on the spatial proximity and functional connectivity of the electrodes; Step S4: The frequency domain features from step S2 and the adjacency matrix obtained in step S3 are input into the attention network for feature aggregation. The extracted feature representation is then input into the classifier to classify the driving intention and output the result. Specifically, step S3 includes: Step S31: Treat the electrode channels as nodes in a spatial structure graph, connect each pair of nodes as edges of the network, use Euclidean distance as the weight value of each edge, and construct a spatial connection matrix by setting a threshold. : In the formula, Represents the spatial connectivity matrix. This represents the Euclidean distance between any two points, and Threshold represents the set distance threshold, which is the average of all distances between electrodes. Step S32: The correlation between each node is calculated using the phase lag index as the weight of each edge to construct the PLI matrix; wherein, the PLI matrix is ​​the phase lag index matrix, which represents the functional connectivity between electrodes by calculating the phase difference between the EEG signals of each electrode. In the formula, express The phase difference of the electrode signal at time point N represents the number of time points, and sign(number) is the sign function. If number is greater than 0, sign returns 1; if number is equal to 0, sign returns 0; if number is less than 0, sign returns -1. Step S33: Binarize the PLI matrix obtained in S32 by setting a threshold to obtain the functional connectivity matrix. The binarization process is as follows: In the PLI matrix, elements greater than the threshold are set to 1, indicating that there is a functional connection between the two electrode EEG signals; elements less than the threshold are set to 0, indicating that there is no functional connection between the two electrode EEG signals. Step S34: Integrate the functional connectivity and spatial proximity of different brain regions to construct the final adjacency matrix. Specifically: In the formula, Represents the final adjacency matrix. Represents the spatial connectivity matrix. Represents the functional adjacency matrix.

2. The driving intention recognition method based on functional connectivity and graph neural networks according to claim 1, characterized in that, Step S1 specifically includes: Step S11: Collect EEG data of the driver under two types of driving behaviors: constant speed driving and braking. The collected data is processed by bandpass filtering from 0.5HZ to 70HZ. Step S12: Remove the artifact components of electrooculography using independent component analysis. Step S13: Remove damaged electrodes and supplement them by interpolation with adjacent electrodes; Step S14: The raw EEG data is divided into non-overlapping time windows of 1 second each, based on the two types of driving intention labels: constant speed driving and braking, and then divided into training set and test set.

3. The method of using the driving intention recognition method based on functional connectivity and graph neural networks according to claim 1, characterized in that, Step S2 specifically includes: Step S21: The EEG signal is decomposed into five bands by fast Fourier transform, namely delta band (0.5-3Hz), theta band (4-7Hz), alpha band (8-13Hz), beta band (14-30Hz) and gamma band (31-70Hz). Step S22: Calculate the power spectral density of the five bands in S21 as the frequency domain characteristics of the EEG signal. The formula for calculating the power spectral density is: In the formula, N represents the power spectral density in the corresponding frequency band; N represents the number of samples. Represents the sampling period; X(w) represents the sampling frequency; X(w) represents the EEG signals of each band in S21.

4. The method of using the driving intention recognition method based on functional connectivity and graph neural networks according to claim 1, characterized in that, Step S4 specifically includes: Step S41: Input the frequency domain features and graph structure into the graph attention network for feature aggregation; Step S42: The feature map obtained after feature aggregation is flattened and read out through a fully connected layer. The read feature representation Z is input into a softmax layer to obtain the probabilities of the two driving intention classification tasks: braking and following. The output result is the task represented by the label encoding with the highest probability value. The softmax layer is specifically as follows: In the formula, Z represents the probability value of the k-th task; Z represents the feature representation of the input; Zk represents; Zc represents; C represents the number of driving intention classification categories; Step S43: Select cross-entropy loss as the loss function, and use the Adam algorithm for gradient descent to optimize the model; the cross-entropy formula is as follows: In the formula, C represents the number of driving intent categories; c represents the category number; The probability distribution of the two driving intention labels, braking and following, output by the classification model. ]; y represents the true category label after one-hot encoding, i.e., the C-dimensional vector after one-hot encoding. This represents the true category label for class c.

5. The method of using the driving intention recognition method based on functional connectivity and graph neural networks according to claim 4, characterized in that, In step S43 The specific calculation is as follows: A classification experiment was conducted with two driving intentions: following another vehicle and braking. The C=2 value was selected, and the true class label for the samples was braking. The labels were obtained after one-hot encoding. It is [0,1]; The true category label of the sample is "following the vehicle," and the label is obtained after one-hot encoding. It is [1,0].

6. The method of using the driving intention recognition method based on functional connectivity and graph neural networks according to claim 4, characterized in that, Step S41 specifically includes: Step S411: Map the features of nodes i and j using vector W respectively, and concatenate the results using a feedforward neural network. Mapping the concatenated vector to real numbers and activating it with the LeakyRELU function yields the importance coefficient of node j to node i. : in, , Represents the feature vectors of nodes i and j; Represents all neighboring nodes of node i; W, This represents the trainable parameters in the model; Step S412, using the adjacency matrix The importance coefficients are updated by reserving the importance coefficient for each node when an element in the adjacency matrix is ​​greater than 0, indicating a connection between the two nodes; otherwise, the coefficient is set to negative infinity. The updated importance coefficients are then normalized using softmax to obtain the relationship between node i and its neighboring nodes j ∈ [the range of nodes i and j]. Attention coefficient : Where k represents the neighboring nodes of node i. Represents all neighboring nodes of node i; Step S413: After obtaining the attention coefficients, feature aggregation is performed on all neighboring nodes j of node i using a weighted average method. in, This represents the feature vector of node i after aggregation; Represent the characteristics of neighbor node j; This represents a vector aggregation function, specifically a weighted average. represents the attention coefficient; W is a trainable parameter.

Images