A brain electrical driving intention prediction method based on a fusion time-frequency feature large model

Abstract

The application provides a driving intention prediction method based on a fusion time-frequency feature large model, which uses an electroencephalogram acquisition system to obtain initial electroencephalogram signals in multiple scenes, and a first Transformer converter and a mask reconstruction mechanism to extract time domain features. After preprocessing and Fbank conversion of the initial electroencephalogram signals, a random mask is applied on the channel, and a second Transformer converter and a random mask strategy are used to extract frequency domain features. The time domain and frequency domain features are fused through a cross-attention mechanism to generate self-learning prompt words. The self-learning prompt words are converted into embedded vectors accepted by a large language model, the large language model outputs driving intention probabilities, and the driving intention corresponding to the maximum probability value is taken as the prediction result. The method fuses time domain features and frequency domain features to generate self-learning prompt words. The application can realize dynamic fusion of time-frequency features, thereby improving the prediction accuracy of the driving intention output by the large language model and improving the reasoning accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electroencephalogram signal processing, and in particular to a driving intention prediction method based on a large model driven by fusion time-frequency features. BACKGROUND

[0002] Nowadays, intelligent driving technology is shifting from single-scene automation execution to deep cognitive collaboration. With the breakthroughs in the integration of large models, vehicle-road cloud, and other technologies, intelligent driving systems not only need to identify physical information such as road markings and obstacles, but also need to build real-time understanding capabilities for the decision-making logic of drivers. In complex scenarios such as mixed driving in urban areas and sudden obstacle avoidance, establishing a "decision consensus" between vehicles and human drivers has become a key factor in expanding the safety boundaries of the system, which requires intelligent driving to accurately capture the cognitive trajectory of the driver before the physical action occurs. Current mainstream technologies mainly rely on vehicle sensors and visual behavior monitoring to infer driving intentions, but they have defects such as response lag and insufficient ability to capture implicit driving intentions. Under this background, neural decoding technology based on EEG (Electroencephalogram) signals has attracted attention because it can directly reflect the neural activity of the driver's cognitive decision-making.

[0003] However, existing solutions still have significant limitations: (1) Single feature: usually only relying on a single modality in time or frequency domain, or integrating both through simple concatenation, ignoring the dynamic correlation of EEG signals in time-frequency dimensions, making it difficult to effectively extract key neural patterns. (2) Insufficient time series modeling capability: traditional RNN and LSTM methods have obvious defects in modeling long-range dependencies, making it difficult to accurately capture continuous intention changes during driving. (3) Subjectivity of large model prompts: the prompts used in large model reasoning usually rely on human design, which is highly subjective, and cannot generate prompts adaptively based on individual EEG features, limiting the performance of large language models in intention understanding. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a driving intention prediction method based on a large model driven by fusion time-frequency features, which realizes the dynamic fusion of time-domain features and frequency-domain features and generates self-learning prompts.

[0005] The technical solution adopted by the present application to solve the above technical problems is: a driving intention prediction method based on a large model driven by fusion time-frequency features, the method comprising the following steps:

[0006] Step 1, data acquisition: obtaining the EEG signals of the driver during the driving process to form the initial EEG signals, or obtaining the EEG signals of the driver during the driving process after preliminary processing to form the initial EEG signals;

[0007] Step 2, time domain feature extraction: the initial electroencephalogram signal is subjected to time dimension random mask, a part of time steps is randomly selected to be covered, a time domain input sequence after the mask is obtained, the time domain input sequence after the mask is taken as the input of the first Transformer converter, and the time domain feature is extracted through the first Transformer converter;

[0008] Step 3, frequency domain feature extraction: the initial electroencephalogram signal of each channel is preprocessed and subjected to Fbank transformation to obtain a frequency domain signal; a mask matrix is generated by randomly selecting part of the data of part of the channels of the frequency domain signal, and a frequency domain input sequence after the mask is obtained, the frequency domain input sequence after the mask is taken as the input of the second Transformer converter, and the frequency domain feature is extracted through the second Transformer converter;

[0009] Step 4, self-learning word generation: the time domain feature and the frequency domain feature are fused by a cross attention mechanism to obtain a self-learning prompt word.

[0010] Preferably, the electroencephalogram driving intention prediction method based on the fusion time-frequency feature large model further comprises the following steps:

[0011] Step 5, large language model reasoning: the self-learning prompt word obtained in step 4 is converted into an embedding vector accepted by a large language model, and is input into the large language model for end-to-end training;

[0012] Step 6, output of the predicted driving intention: the large language model outputs a driving intention probability, and the driving intention corresponding to the maximum probability value is taken as the prediction result.

[0013] Preferably, in the step 2, the initial electroencephalogram signal is subjected to time dimension random mask to obtain a time domain input sequence after the mask .

[0014] , wherein , m i represents the mask state of the i-th time step;

[0015] The time domain input sequence after the mask is . , and represents element-wise multiplication, .

[0016] Preferably, the first Transformer converter comprises a first encoder and a first decoder; and in the step 2, the step of extracting the time domain feature by the first Transformer converter comprises the following steps:

[0017] 201), the time domain input sequence after the mask is input into the first encoder, and the first encoder outputs a feature h;

[0018] ;

[0019] wherein: d is the encoding dimension;

[0020] 202), input of the first decoder is set as follows:

[0021] ;

[0022] ;

[0023] ;

[0024] wherein, is an embedding representation of the mask state, is a learnable embedding vector,

[0025] is a projection weight matrix, denotes a feature concatenation operation, is a reconstructed electroencephalogram sequence output by the first encoder;

[0026] 203), only calculate the MSE Loss for the masked time steps,

[0027] temporal reconstruction loss:

[0028] , is a set of time steps;

[0029] 204), take the hidden features of the last layer of the first encoder as the temporal features .

[0030] Preferably,

[0031] In step 3, the initial electroencephalogram signals under multiple driving states are obtained, and the signals of each channel are sequentially preprocessed and Fbank transformed to obtain the feature , denotes the Fbank feature matrix of the cth channel;

[0032] Randomly select part of the data of the masked part of the channel to generate a mask matrix ; ; ; ; denotes the feature value of the cth channel, the tth time frame, and the fth frequency band;

[0033] masked frequency domain input sequence ; ⊙ denotes element-wise multiplication.

[0034] Preferably,

[0035] In the step 3, the second Transformer converter comprises a second encoder and a second decoder;

[0036] The step of extracting the frequency domain feature by the second Transformer converter comprises the following:

[0037] 301), performing linear transformation on the frequency domain input sequence after the mask to obtain an embedded sequence ;

[0038] ;

[0039] The input with position information , P is position encoding; the input with position information as the input of the second encoder;

[0040] wherein, denotes a frequency domain embedding weight matrix, denotes a frequency domain embedding bias vector, and is a bias term added to the linear transformation;

[0041] 302), the feature h2 output by the second encoder is input into the second decoder; , the frequency domain reconstruction signal output by the second decoder;

[0042] 303), only the channel data that is masked is used to calculate the MSE Loss;

[0043] The frequency domain reconstruction loss is:

[0044] ,

[0045] is a mask position set;

[0046] 304), frequency domain feature extraction: taking the hidden feature output by the last layer of the second encoder as the frequency domain feature .

[0047] Preferably,

[0048] In the step 4, the step of fusing the time domain feature and the frequency domain feature to obtain the self-learning prompt word by the cross-attention mechanism comprises the following:

[0049] The time domain feature and the frequency domain feature ​Cross attention fusion is performed, time domain features are taken as queries, frequency domain features are taken as keys and values, and time-frequency relationship is learned;

[0050] Q, K, V are defined, representing Query, Key, Value respectively;

[0051] ; ; ;

[0052] Q represents a query projection weight matrix, and the core focus of the time domain feature is learned;

[0053] K represents a key projection weight matrix, and the matching key information of the frequency domain feature is extracted;

[0054] V represents a value projection weight matrix, and the fusible content of the frequency domain feature is organized;

[0055] The cross attention weight is calculated:

[0056] ; ;

[0057] A represents the attention degree of each time domain position to each frequency domain position; is the dimension within the attention;

[0058] is the transpose of the key matrix;

[0059] Generate fusion features :

[0060] ;

[0061] The time domain features extract key information into the frequency domain features to obtain fusion features , and the fusion features are taken as self-learning prompt words.

[0062] Preferably, the EEG driving intention prediction method based on the fusion time-frequency feature large model further comprises

[0063] 501), the self-learning prompt word is converted into a token sequence accepted by Mistral,

[0064] and [CLS] token is added;

[0065] ;

[0066] 502), input the features into the pre-trained Mistral architecture;

[0067] ;

[0068] Wherein each token outputs a hidden layer representation.

[0069] 503), global feature aggregation, the large language model outputs a sequence z,

[0070] ;

[0071] T is the length of the sequence, represents the hidden layer representation of the tth token;

[0072] Preferably,

[0073] The large language model comprises an MLP multilayer perceptron, a self-learning prompt word inputting the MLP multilayer perceptron, a softmax classification, and a predicted result obtained;

[0074] ;

[0075] Wherein, Is a learnable weight, b is a bias; c represents the number of categories of driving intention classification; a softmax output prediction probability distribution, Is a predicted result; Represents a classification bias vector; Represents a self-learning prompt word feature vector.

[0076] Preferably,

[0077] The loss function in the large language model uses a cross-entropy loss:

[0078] ;

[0079] Represents the i-th component of the true label,

[0080] Represents the i-th predicted probability.

[0081] The beneficial effects of the present application are: the present application is a kind of based on fusion time-frequency characteristic big model driving electroencephalogram driving intention prediction method, in time domain introduces random mask strategy, combine first Transformer converter and mask reconstruction mechanism extracts deep time domain characteristics. The second Transformer converter and random mask strategy are used to extract frequency domain features. Cross attention mechanism is used to fuse time domain and frequency domain features to generate self-learning prompt words. The present application can realize the dynamic fusion of time-frequency characteristics, and generate self-learning prompt words to input into the large language model, thereby improving the prediction accuracy of the large language model output driving intention, improving the inference precision, adaptability and practical application value. Attached Figure Description

[0082] Figure 1 This is a flowchart of the EEG driving intention prediction method based on a large model of fused time-frequency features according to the present invention.

[0083] Figure 2 This is a configuration diagram of each layer of the first encoder in this invention;

[0084] Figure 3 This is a configuration diagram of each layer of the first decoder in this invention;

[0085] Figure 4 This is a configuration diagram of each layer of the second encoder in this invention;

[0086] Figure 5 This is a configuration diagram of each layer of the second decoder in this invention. Detailed Implementation

[0087] The present invention will now be described in further detail with reference to the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0088] The present invention will now be described in further detail with reference to the accompanying drawings and preferred embodiments. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0089] like Figure 1 As shown, a method for predicting driving intentions based on a large model driven by fused time-frequency features is presented. This method includes the following steps:

[0090] Step 1, Data Acquisition: Acquire the driver's EEG signals during multiple driving scenarios to form an initial EEG signal, or acquire the driver's EEG signals during multiple driving scenarios and perform preliminary processing to form an initial EEG signal; for example, use a standard 32-channel EEG acquisition system at a sampling frequency of 256Hz to acquire the initial EEG signals under multiple driving scenarios. , X t This represents the EEG signal at time step t, where t is the number of time steps (e.g., 256), and C represents the number of signal channels (e.g., 32 channels). Preliminary processing of the EEG signal includes denoising, artifact removal, and standardization. The initial EEG signal is a time-domain signal.

[0091] Step 2, time domain feature extraction: the initial electroencephalogram signal is randomly masked in the time dimension, a part of the time steps is randomly selected to be masked, and a masked time domain input sequence is obtained. The masked time domain input sequence is input into a first Transformer converter to extract time domain features.

[0092] Step 3, frequency domain feature extraction: the initial electroencephalogram signal of each channel is preprocessed and subjected to Fbank transformation to obtain a frequency domain signal; a mask matrix is generated by randomly selecting part of the data of part of the channels of the frequency domain signal, and a masked frequency domain input sequence is obtained. The masked frequency domain input sequence is input into a second Transformer converter to extract frequency domain features.

[0093] Step 4, self-learning word generation: cross-attention mechanism is used to fuse the time domain features and the frequency domain features to obtain a self-learning prompt word.

[0094] Step 5, large language model inference: the self-learning prompt word is converted into an embedding vector accepted by a large language model, and is input into the large language model for end-to-end training.

[0095] Step 6, output predicted driving intention: the large language model outputs a driving intention probability, and the driving intention corresponding to the maximum probability value is taken as the prediction result.

[0096] In the step 2, the initial electroencephalogram signal is randomly selected to mask part of the time steps to generate a mask vector ; for example, 30% of the time steps are randomly selected to be masked. , wherein , m i represents the mask state of the i-th time step (0 = masked, 1 = retained).

[0097] The masked time domain input sequence is ; , and represents element-wise multiplication. .

[0098] Specifically, in an implementable scheme, the first Transformer converter includes a first encoder and a first decoder. In the step 2, the step of extracting time domain features by the first Transformer converter includes the following steps:

[0099] 201), the masked time domain input sequence is input into the first encoder, and the first encoder outputs a feature h;

[0100] ;

[0101] wherein: d is the encoding dimension, which is 64.

[0102] 202), input of the first decoder is set as follows:

[0103] ;

[0104] ;

[0105] ;

[0106] wherein, is the embedded representation of the mask state, is the learnable embedding vector,

[0107] is the projection weight matrix, denotes the feature concatenation operation, is the reconstructed EEG signal sequence output by the first encoder.

[0108] 203), only calculate MSE Loss (Mean Square Error Loss) for the masked time steps,

[0109] define the set of time steps: ;

[0110] then the time domain reconstruction loss is:

[0111] ;

[0112] 204), take the hidden features of the last layer of the first encoder as the time domain features .

[0113] the reconstructed EEG signal sequence output by the first encoder is compared with the original unmasked signal X, mainly to give the model a self-supervised training target, that is, the reconstruction loss (MSE Loss). is only used to calculate the loss in the training stage, and will not flow into the subsequent feature extraction. The hidden features output by the last layer of the first encoder are still used by the downstream.

[0114] Advantages: By introducing a time-dimension random mask mechanism, the missing or interference phenomenon in the electroencephalogram signal can be simulated at the time domain level, enhancing the robustness of the model to abnormal data. At the same time, this strategy forces the model to rely on context information to reconstruct the masked time steps, effectively learning global temporal dependencies. Compared with traditional sequence modeling without masks, this method can further suppress local overfitting, improve feature discriminability and generalization ability, and provide structural compatibility for subsequent alignment with large language models.

[0115] The mask and reconstruction mechanism randomly masks part of the input signal and trains the model to restore it only relying on the unmasked part, thereby achieving self-supervised learning. This process does not require human annotation and effectively utilizes massive raw signal data. At the same time, the model is forced to learn the context dependence and structural rules of the time series signal during the reconstruction process, significantly improving the robustness to missing data and noise interference. In addition, this mechanism also has a certain regularization effect, which can prevent the model from overfitting to local patterns and help to learn more generalizable feature representations.

[0116] Specifically, in an implementable scheme,

[0117] In step 3, the initial electroencephalogram signal under multiple driving conditions is obtained, and the signals of each channel are sequentially pre-emphasized, framed, Fourier transform power spectrum calculated, and Fbank transformed to obtain features , Fbank feature matrix of the cth channel, , c = 32.

[0118] Randomly select part of the data of the masked part of the channel to generate a mask matrix ; for example, randomly select 30% of the data of one channel to mask; ; ; ; Fbank feature matrix of the cth channel,

[0119] Masked frequency domain input sequence ; , ⊙ represents element-wise multiplication.

[0120] Specifically, in an implementable scheme,

[0121] In step 3, the second Transformer converter includes a second encoder and a second decoder.

[0122] The steps of the second Transformer converter extracting frequency domain features include the following:

[0123] 301), the frequency domain input sequence after masking Linear transformation to get the embedding sequence

[0124]

[0125] Input with position information P is the position encoding; input with position information as the input of the second encoder;

[0126] where, denotes the frequency domain embedding weight matrix, denotes the frequency domain embedding bias vector, which adds a bias term to the linear transformation; .

[0127] 302), the feature h2 output by the second encoder as the input of the second decoder; , the frequency domain reconstruction signal output by the second decoder where, is the complete sequence predicted by the second decoder, and the masked part is filled by the second decoder;

[0128] 303), loss function (MSE Loss): only calculate MSE Loss for the masked channel data;

[0129] Frequency domain reconstruction loss:

[0130] ,

[0131] is the set of mask positions;

[0132] the frequency domain reconstruction signal output by the second decoder and the unmasked signal for comparison, calculate the reconstruction loss.

[0133] 304), frequency domain feature extraction: take the hidden feature output by the last layer of the second encoder as the frequency domain feature

[0134] ​​​​​​Through the random mask strategy of the channel dimension, the adaptability of the model to band interference, absence of specific channel signals and the like can be effectively enhanced, and the robustness of frequency domain feature modeling is improved. Meanwhile, the strategy prompts the model to mine global correlations in the spectral structure under the condition of partial band information loss, which helps to improve the modeling capability of inter-band context dependence and suppresses the overfitting of the model to a single frequency region. In addition, the mask mechanism is symmetrical in structure with the time domain mask, providing a unified and robust feature expression basis for subsequent time-frequency feature fusion and prompt word generation.

[0135] Specifically, in an implementable scheme,

[0136] In step 4, the step of obtaining a self-learning prompt word by fusing time domain features and frequency domain features through a cross-attention mechanism includes the following:

[0137] Time domain features and frequency domain features are cross-attention fused, with the time domain features as queries, the frequency domain features as keys and values, and the time-frequency relationship is learned;

[0138] Define Q, K, and V, representing Query, Key, and Value, respectively.

[0139] ; ; ;

[0140] Q represents the query projection weight matrix, which learns the core focus of the time domain features.

[0141] K represents the key projection weight matrix, which extracts the matching key information of the frequency domain features.

[0142] V represents the value projection weight matrix, which organizes the fusible content of the frequency domain features.

[0143] Calculate the cross-attention weight:

[0144] ; ;

[0145] A represents the attention degree of each time domain position to each frequency domain position. is the dimension within the attention;

[0146] is the transpose of the key matrix.

[0147] Generate fusion features :

[0148] ;

[0149] Fusing the time domain features and the frequency domain features to obtain a fusion feature , the fusion feature as a self-learning prompt word.

[0150] Compared with the existing large language model reasoning method relying on artificial setting of prompt words, the self-learning prompt word generated by the present application has stronger objectivity and adaptability. The traditional prompt word has significant artificial subjectivity and cannot be dynamically adjusted according to individual electroencephalogram features, which limits the generalization ability and accuracy of intention understanding. The present application generates a prompt word by fusing the time domain and frequency domain features of the electroencephalogram signal, which can realize high matching between the prompt content and the state of the measured individual, thereby significantly improving the understanding ability and output performance of the large language model in driving intention prediction.

[0151] Specifically, in an implementable scheme,

[0152] Step 5: input the self-learning prompt word into the large model. Convert the self-learning prompt word into Token Embedding accepted by the LLM large language model Mistral, and input it into the large model and perform end-to-end fine-tuning.

[0153] 501), self-learning prompt word converted into a token sequence accepted by Mistral

[0154] and add a [CLS] token;

[0155] ;

[0156] 502), input the features into the pre-trained Mistral architecture (fine-tuning);

[0157] ;

[0158] Each token outputs a hidden layer representation.

[0159] 503), global feature aggregation, the large language model outputs a sequence, each token corresponds to a vector representation, in order to make a prediction, the sequence is aggregated into a fixed-length representation, using global average pooling (Global Average Pooling) can effectively fuse the position information and avoid over-reliance on a single position, which can be mathematically represented as:

[0160] ;

[0161] T is the length of the sequence (including [CLS]), and represents the hidden layer representation of the tth token.

[0162] Specifically, in an implementable solution,

[0163] Step 6, the large language model includes an MLP multilayer perceptron, the self-learning prompt word input MLP multilayer perceptron, and the prediction result is obtained through softmax classification;

[0164]

[0165] wherein, is a learnable weight, b is a bias; c represents the number of driving intention classification categories (such as left turn, right turn, straight, etc.); softmax output prediction probability distribution, is the prediction result; represents a classification bias vector;

[0166] represents a self-learning prompt word feature vector.

[0167] Specifically, in an implementable solution,

[0168] The loss function in the large language model uses cross-entropy loss:

[0169]

[0170] represents the i-th component of the true label,

[0171] represents the i-th predicted probability;

[0172] The cross-entropy loss is used to further back-propagate and gradually reduce the loss value, and the parameters of the training input projection layer and the classifier head are optimized.

[0173] ​​The application provides a driving intention prediction method based on a fusion time-frequency feature large model driven electroencephalogram, and an electroencephalogram acquisition system is used to obtain initial electroencephalogram signals in multiple scenes, which can be optionally preliminarily processed. Then, a random mask strategy is introduced, a first Transformer converter and a mask reconstruction mechanism are combined to extract deep time domain features. The initial electroencephalogram signals are subjected to Fbank conversion, and a second Transformer converter and a random mask strategy are used to extract frequency domain features. Then, the time domain and frequency domain features are fused through a cross attention mechanism to generate self-learning prompt words. In the reasoning stage, the self-learning prompt words are mapped into Token sequences and added with a [CLS] label for end-to-end training. Finally, an MLP multilayer perceptron and a SoftMax layer are used to output the final driving intention. The method fuses time domain features and frequency domain features to generate self-learning prompt words, and the application can realize dynamic fusion of time-frequency features, thereby improving the prediction accuracy of the driving intention output by the large language model and improving the reasoning accuracy.

[0174] The above description is only a specific embodiment of the application, and various examples do not limit the essential content of the application, and those skilled in the art can modify or deform the above described specific embodiments without departing from the essence and scope of the application.

Claims

1. A driving intention prediction method based on a fusion time-frequency feature large model, characterized in that: The method comprises the following steps: Step 1, data acquisition: obtaining the brain electrical signals of the driver in the multi-scene driving process to form initial brain electrical signals, or obtaining the brain electrical signals of the driver in the multi-scene driving process to form initial brain electrical signals after preliminary processing; Step 2, time domain feature extraction: performing time dimension random mask on the initial brain electrical signals, randomly selecting a part of time steps to be masked, obtaining a time domain input sequence after masking, taking the time domain input sequence after masking as the input of a first Transformer converter, and extracting time domain features through the first Transformer converter; Step 3, frequency domain feature extraction: performing preprocessing and Fbank conversion on the initial brain electrical signals of each channel to obtain frequency domain signals; randomly selecting a part of data of a part of channels to generate a mask matrix to obtain a frequency domain input sequence after masking, taking the frequency domain input sequence after masking as the input of a second Transformer converter, and extracting frequency domain features through the second Transformer converter; Step 4, self-learning word generation: fusing the time domain features and the frequency domain features through a cross-attention mechanism to obtain self-learning prompt words; The time domain feature extracts key information from the frequency domain feature to obtain a fusion feature , the fusion feature as a self-learning prompt word; Step 5, large language model reasoning: converting the self-learning prompt words obtained in step 4 into embedded vectors accepted by a large language model, and inputting the embedded vectors into the large language model for end-to-end training; Step 6, outputting a predicted driving intention: taking the driving intention corresponding to the maximum probability value as the prediction result.

2. The method of claim 1, wherein the method is based on a fusion time-frequency feature large model driven electroencephalogram driving intention prediction. In step 2, the initial electroencephalogram signal Randomly select the mask part time step, generate a mask vector ; wherein , m i denotes the mask state at the i-th time step; The time-domain input sequence after the mask is ; , ⊙ represents element-wise multiplication, .

3. The method of claim 2, wherein the method is based on a fusion time-frequency feature large model driven electroencephalogram driving intention prediction method. The first Transformer converter comprises a first encoder and a first decoder; in step 2, the step of extracting time domain features by the first Transformer converter comprises the following: 201) the time-domain input sequence after masking an input first encoder, the first encoder outputting a feature h; ； wherein: d is the coding dimension; 202) input of the first decoder is set as follows: ； ； ； wherein, is an embedded representation of the mask state, is a learnable embedding vector, is a projection weight matrix, denotes a feature concatenation operation, is a reconstructed electroencephalography signal sequence output by the first encoder; 203), only calculating the MSE Loss for the masked time steps, Time domain reconstruction loss: , is a set of time steps; 204)、take the hidden features of the last layer of the first encoder as the time domain features .

4. The brain electrical driving intention prediction method based on a large model driven by fused time-frequency features according to claim 1, characterized in that: In step 3, the initial electroencephalogram signals in multiple scene driving states are acquired, and the signals of each channel are sequentially preprocessed and Fbank transformed to obtain features , Fbank feature matrix of the cth channel is represented Generating a mask matrix by randomly selecting portions of data that mask portions of channels ; ; ; ; denotes the eigenvalue of the cth channel, the tth time frame, the fth frequency band Masked frequency-domain input sequence ; , denotes element-wise multiplication.

5. The brain electrical driving intention prediction method based on a large model driven by fused time-frequency features according to claim 4, characterized in that: In step 3, the second Transformer converter comprises a second encoder and a second decoder; The step of extracting frequency domain features by the second Transformer converter comprises the following: 301), the frequency domain input sequence after masking performing linear transformation to obtain the embedding sequence ; ； Input with position information P is a position code; input with position information as input to a second encoder wherein, denotes a frequency domain embedding weight matrix, denotes a frequency domain embedding bias vector, adding a bias term to the linear transformation; 302) the feature h2 output by the second encoder as input to the second decoder; the frequency domain reconstructed signal output by the second decoder ; 303), only calculating the MSE Loss for the masked channel data; Frequency domain reconstruction loss: ， mask position set; 304), frequency domain feature extraction: take the hidden features of the last layer output of the second encoder as the frequency domain features .

6. The brain electrical driving intention prediction method based on a large model driven by fused time-frequency features according to claim 1, characterized in that: In step 4, the step of fusing the time domain features and the frequency domain features through a cross-attention mechanism to obtain self-learning prompt words comprises the following: time domain features and frequency domain features cross-attention fusion is performed, the time domain features are taken as queries, the frequency domain features are taken as keys and values, and a time-frequency relationship is learned; Defining Q, K, and V, respectively representing Query, Key, and Value; ； ； ； represents the query projection weight matrix, the core focus of learning the time domain features; represents the key projection weight matrix, and the matching key information of the frequency domain feature is extracted; represents a value projection weight matrix, which organizes the fusable content of the frequency domain features; Calculating the cross-attention weight: ； ； A represents the degree of attention to each frequency domain position for each time domain position; is the dimension within attention; is the transpose of the key matrix; Generating fused features : 。 7. The method according to claim 1 or 6, wherein the method is characterized in that: Further comprising 501) self-learning cue words converted into a sequence of tokens accepted by Mistral, And adding the [CLS] token; ； 502), inputting the input features into a pre-trained Mistral architecture; ； Each token outputs a hidden layer representation; 503), global feature aggregation, the large language model outputs a sequence z, ； T is the length of the sequence, representing the hidden layer representation of the tth token.

8. The method according to claim 1, wherein the method is characterized in that: The large language model comprises an MLP multilayer perceptron, and the self-learning prompt word input is input into the MLP multilayer perceptron, and a prediction result is obtained through softmax classification. ； wherein, are learnable weights, b is a bias; c represents the number of categories of driving intention classification; softmax outputs a prediction probability distribution, is a prediction result; represents a classification bias vector; represents a self-learning prompt word feature vector.

9. The method according to claim 8, wherein the method is characterized in that: The loss function in the large language model uses a cross-entropy loss. ； Represents the i-th component of the true label. represents the predicted i-th class probability.

Images