A subject-specific task assessment method

Abstract

The application discloses a subject individual-oriented suitable task evaluation method for BCI system optimization. The application collects EEG data of motor imagination and speech imagination from a single subject by designing a specific experimental paradigm. By using a cross-task decoding technology, the data is converted into different perspectives of electroencephalogram, so as to extract key electroencephalogram features related to different tasks. Then, by constructing a deep learning network containing multiple convolution and pooling layers, the features are further analyzed, the performance of the model is evaluated, and the performances of different tasks in multiple performance indicators are compared. The application comprehensively uses multi-task EEG data collection, cross-task decoding technology and a deep learning network model to accurately analyze and determine the most suitable task type for the subject.

Description

Technical Field

[0001] This invention relates to a task assessment method applicable to individual subjects, belonging to the field of neural information decoding. Background Technology

[0002] With advancements in neuroscience research and the development of computer technology, brain-computer interface (BCI) technology has become an important branch of neuroengineering. The core objective of BCI technology is to establish a non-muscle-controlled communication pathway directly between the brain and an external device to assist or enhance human cognitive or motor functions. In practical applications, BCI systems often need to process complex EEG data from a single subject participating in multiple tasks (motor imagery tasks and verbal imagery tasks). Effective analysis of this cross-task EEG data allows for precise identification of the subject's brain activity patterns under different tasks. To improve the performance of the BCI system, this invention employs a multi-perspective feature decoding network (spatial connectivity perspective, spatiotemporal perspective, and time-frequency perspective) to decode and analyze EEG data. This method not only enhances the model's adaptability to different EEG signal features but also, by comprehensively considering multiple key performance indicators such as decoding accuracy, training time, and testing time, can recommend the most suitable task type for each subject. Summary of the Invention

[0003] This invention proposes a task assessment method applicable to individual subjects, which is used to optimize EEG data processing and task decoding for single subjects in a multi-task environment.

[0004] The technical solution adopted in this invention is: a task assessment method applicable to individual test subjects, comprising the following steps:

[0005] (1) Construct different experimental paradigms based on different tasks and extract EEG data for different tasks;

[0006] (2) By using a multi-view feature extraction network, EEG data is converted into different perspectives of EEG and key EEG features related to the task are extracted.

[0007] (3) Construct a deep learning network model, extract features of different types of time-frequency maps, and decode EEG features;

[0008] (4) For the above deep learning network model, EEG data is used to evaluate the cross-task analysis performance of the model.

[0009] Specifically, in step (1), the sampling rate is set to 500Hz, and a 60Hz notch filter is used to eliminate interference, as shown in the following formula:

[0010]

[0011] Among them, w n S is the notch frequency, k1 is the notch coefficient 1, k2 is the notch coefficient 2, and S is the input signal frequency.

[0012] Specifically, step (2) includes:

[0013] 1) Constructing a spatial connectivity perspective

[0014] Calculate the correlation between signals from different brain regions:

[0015]

[0016] Among them, C ij X is the correlation coefficient between the i-th and j-th electrodes. i,t and X j,t These are the voltage values ​​of electrodes i and j at time point t, respectively. and This is the corresponding average voltage value, and T represents the total number of time points.

[0017] The calculated connectivity value C ij This forms a connectivity matrix, where each element of the matrix represents the connection strength between a pair of electrodes;

[0018] 2) Constructing a spatiotemporal perspective

[0019] a time series analysis

[0020] Time-domain features were extracted using the mean and variance of EEG data.

[0021]

[0022]

[0023] Where, x i,t Mean is the input signal value of the i-th electrode at time point t. i It calculates the average voltage of the i-th electrode over a given time period, Variance. i This represents the variance used to calculate the signal of the i-th electrode;

[0024] b-space pattern analysis

[0025] Spatial filtering analysis was used to evaluate the distribution of EEG signals among different electrodes on the scalp.

[0026]

[0027] Among them, y i It is the signal value of the i-th electrode after Laplace filtering, x iIt is the original signal value, and Neighbours(i) represents the neighboring electrodes of the i-th electrode;

[0028] 3) Constructing a time-frequency perspective

[0029] Short-time Fourier transform is used to evaluate the local frequency characteristics of a signal:

[0030]

[0031] Where, x i (τ) is the signal of the i-th electrode, and ω(t-τ) is a window function used to localize the signal at time t.

[0032] Specifically, in step (3), a convolutional neural network consisting of 4 convolutional layers, 2 pooling layers, and 2 fully connected layers is constructed to extract features from time-frequency maps of different categories. Let X... i Let i = (1,...,n) be n training set samples. The convolutional layer receives the input data, extracts the sample features, and obtains a series of feature maps:

[0033] feature map =conv(X) i (7)

[0034] In the above formula, X i For training samples, KS represents the size of the convolutional kernel, KN represents the number of convolutional kernels, and conv(*) is the defined convolutional layer;

[0035] The ReLU activation function is used to make the output of each network layer non-linear:

[0036] f(x) = max(0,x) (8)

[0037] x represents the input parameter. After convolution, activation, and pooling, the feature map is input into a classifier composed of fully connected layers for classification. Each fully connected layer consists of multiple neurons, and the layers are fully connected to each other. The neurons in each layer are independent, and the expression for a single neuron is:

[0038]

[0039] Among them, W i Let b represent the weights and b represent the bias. The last layer in the fully connected layer is activated using the softmax function.

[0040]

[0041] In the above formula, p i Let z represent the probability of class i. iThis indicates that the network output for class i is calculated, cross-entropy is used as the loss function, the Adam optimizer is selected, and the optimizer learning rate is set to 3×10. -4 The cross-entropy loss function is defined as follows:

[0042]

[0043] Among them, y i p represents the true label of category i. i Let N represent the probability of class i, N be the total number of training samples, and k be the number of classes.

[0044] Specifically, step (4) decodes and analyzes the cross-task EEG of a single subject based on a multi-view feature extraction network and a deep learning network model, determines the applicable task type for the single subject based on the analysis results, and scores the subject's EEG performance on different tasks:

[0045]

[0046]

[0047] Where K is the reference standard for decoding accuracy, a is the accuracy of task A, and T is the accuracy of the decoding. train The standard for representing training time, T test A reference standard for indicating test time. This represents the training time for task A. This indicates the test time for task A. This indicates the training time for task B. This represents the testing time for Task B. The weighting coefficients for different indicators are X1 + X2 + X3 = 1. After calculating the evaluation score, the applicable task decision output is based on the evaluation score.

[0048]

[0049] When S A >S B When the applicable task's decision result is A, and S... B >S A When the applicable task's decision outcome is B, if S A <0 and S B If the value is less than 0, then recalibrate.

[0050] Beneficial effects:

[0051] This invention introduces a multi-perspective feature adaptation method, enabling the BCI system to accurately analyze and decode multi-perspective features of different tasks such as motor imagery and verbal imagery, achieving efficient cross-task EEG decoding. The use of a convolutional neural network configuration effectively extracts key information from EEG data. Furthermore, this invention compares performance metrics such as decoding accuracy, training time, and testing time in different task executions, and uses a specific algorithm to calculate the most suitable task type for the subject. Attached Figure Description

[0052] Figure 1 It is a task assessment scheme applicable to individual test subjects;

[0053] Figure 2 It is a multi-task EEG data acquisition experiment for subjects;

[0054] Figure 3 This is a graph showing how the accuracy of motor imagery and verbal imagery changes over a training cycle;

[0055] Figure 4 It is an analysis of training and testing times for different tasks;

[0056] Figure 5 These are the evaluation scores for different tasks. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1: As Figure 1-5 As shown, a task assessment method applicable to individual test subjects includes the following steps:

[0059] 1) This study uses different experimental paradigms to extract EEG data from different tasks.

[0060] 2) A multi-view feature extraction network was introduced to convert the EEG data generated in step (1) into different EEG perspectives (spatial connectivity perspective, spatiotemporal perspective, and time-frequency perspective) to extract key EEG features related to the task.

[0061] 3) Construct a deep learning network model, extract features from different types of time-frequency maps, and decode the EEG features.

[0062] 4) Based on the described deep learning network model, evaluate the model's cross-task analysis performance using existing motor imagery and verbal imagery EEG data. By comparing the performance of the two tasks across multiple metrics, select the task type that performs best under the current experimental setup.

[0063] The specific implementation process is as follows:

[0064] This embodiment uses the NeuroScan 64-electrode system to collect EEG data for motor and verbal imagery according to the 10 / 20 international standard, employing a 500Hz sampling rate and a 60Hz notch filter to handle interference. The EEG data is converted into feature inputs based on spatial connectivity, spatiotemporal, and time-frequency perspectives. A convolutional neural network is constructed, and ReLU and softmax layers are used for classification. Training is optimized using the Adam optimizer and cross-entropy loss function. The network model is evaluated on an independent test set, and task scores are calculated based on metrics such as decoding accuracy, training time, and testing time. Finally, the most suitable task type for the subject is determined and recommended based on the comprehensive score.

[0065] Furthermore, step 1 first constructs different experimental paradigms based on different tasks (such as...). Figure 2 (As shown). Before applying task assessment to subjects, it is necessary to collect EEG data from individual subjects in different task scenarios. In this acquisition experimental protocol, the sampling rate was set to 500Hz, and a 60Hz notch filter was used to eliminate interference, as shown in the following formula:

[0066]

[0067] Among them, w n S is the notch frequency, k1 is the notch coefficient 1, k2 is the notch coefficient 2, and S is the input signal frequency.

[0068] Furthermore, step 2 introduces a multi-view feature extraction network (such as...) Figure 1 By converting EEG data into different perspectives of EEG (spatial connectivity perspective, spatiotemporal perspective, and time-frequency perspective), key EEG features related to the task can be extracted.

[0069] 1) Constructing a spatial connectivity perspective

[0070] a. Functional connectivity

[0071] Functional connectivity is typically assessed by calculating the correlation between signals from different brain regions. Below is the expression for the correlation:

[0072]

[0073] Among them, C ij X is the correlation coefficient between the i-th and j-th electrodes. i,t and Xj,t These are the voltage values ​​of electrodes i and j at time point t, respectively. and This represents the corresponding average voltage value, and T represents the total number of time points.

[0074] The connectivity value C obtained using the above method ij This can form a connectivity matrix, where each element represents the connection strength between a pair of electrodes. This matrix can be used for further network analysis and to calculate various graph theory metrics.

[0075] 2) Constructing a spatiotemporal perspective

[0076] a. Time series analysis

[0077] Converting EEG data into a spatiotemporal perspective involves a comprehensive analysis of temporal and spatial information. Time series analysis primarily focuses on how signals change over time, and various time-domain feature extraction techniques can be used. The following uses mean and variance:

[0078]

[0079]

[0080] Where, x i,t Mean is the input signal value of the i-th electrode at time point t. i It calculates the average voltage of the i-th electrode over a given time period, Variance. i This represents the variance used to calculate the i-th electrode signal.

[0081] b. Spatial Pattern Analysis

[0082] Spatial pattern analysis can be used to evaluate the distribution of EEG signals among different electrodes on the scalp, and spatial filtering can be employed.

[0083]

[0084] Among them, y i It is the signal value of the i-th electrode after Laplace filtering, x i It is the original signal value, and Neighbours(i) represents the neighboring electrodes of the i-th electrode.

[0085] By integrating time series and spatial pattern analysis, EEG data can be comprehensively analyzed from both spatiotemporal dimensions, providing a perspective for cross-task decoding technology.

[0086] 3) Constructing a time-frequency perspective

[0087] To analyze the complex time-frequency interactions in EEG data, the Short Time Fourier Transform (STFT) time-frequency analysis method is employed. It combines information from the time and frequency domains and evaluates the local frequency characteristics of the signal by applying Fourier transform within a short time window, serving as a transformed perspective for the input signal.

[0088] STFT{x i}(t,ω)=∫x i (τ)ω(t-τ)e -jωt dt (6)

[0089] Where, x i (τ) is the signal of the i-th electrode, and ω(t-τ) is a window function used to localize the signal at time t.

[0090] Further, step (3) involves constructing a deep learning network model.

[0091] A convolutional neural network consisting of 4 convolutional layers, 2 pooling layers, and 2 fully connected layers was constructed to extract features from different categories of time-frequency maps for EEG decoding of motor and verbal imagery. Let X... i Let i = (1,...,n) be n training set samples. The convolutional layer receives the input data, extracts the sample features, and obtains a series of feature maps:

[0092] feature map =conv(X) i (7)

[0093] In the above formula, X i For training samples, KS represents the size of the convolution kernel, KN represents the number of convolution kernels, and conv(*) is the defined convolutional layer.

[0094] Activation functions enable non-linearity in the output of each network layer. Generally, ReLU is used as the activation function because it converges quickly during network training and is easy to calculate gradients. The expression for ReLU is as follows:

[0095] f(x) = max(0,x) (8)

[0096] 'x' represents the input parameter. After convolution, activation, and pooling, the feature map is input into a classifier composed of fully connected layers for classification. A fully connected layer consists of multiple neurons, fully connected to each other, and the neurons in each layer are independent. The following is the expression for a single neuron:

[0097]

[0098] Among them, W i'b' represents the weights, and 'b' represents the bias. Similarly, each neuron in a fully connected layer also needs an activation function, but in the last layer, ReLU activation will not be used; instead, the softmax function will be used.

[0099]

[0100] In the above formula, p i Let z represent the probability of class i. i This represents the network output for class i. Using cross-entropy as the loss function, the optimizer "Adam" is chosen to effectively prevent gradient vanishing and improve model performance. The optimizer learning rate is set to 3 × 10⁻⁶. -4 The cross-entropy loss function is defined as follows:

[0101]

[0102] Among them, y i p represents the true label of category i. i Let N represent the probability of class i, N be the total number of training samples, and k be the number of classes.

[0103] Further, step (4) evaluates the cross-task analysis performance of the model using existing motor imagery and verbal imagery EEG data. This is achieved by comparing performance on multiple metrics across the two tasks (e.g., Figure 3 , Figure 4 Select the task type that performs best under the current experimental settings.

[0104] After acquiring multi-task EEG data for a single subject, the cross-task EEG of the single subject is decoded and analyzed based on the aforementioned multi-view feature extraction network and deep learning network model. The appropriate task type for the single subject is determined based on the analysis results. Based on the design performance metrics of the BCI system, this scheme intends to consider three indicators: decoding accuracy, training time, and testing time.

[0105] To calculate the EEG response of a single subject performing different tasks, in addition to the results of the isomorphic cross-task decoding model, we also need to know the baseline indicators for BCI system development. For example, the BCI recognition accuracy under a specific task should not be less than 80%. Subtracting 80% from the actual task's decoding accuracy gives the subject's absolute score on the task's decoding accuracy analysis indicator. Considering standard normalization, the absolute score needs to be converted to a relative score based on the distribution range of the actual accuracy. Other indicators are also evaluated using the same scoring method. In calculating the global score for task suitability, since different BCI systems for different purposes have different requirements for different performance indicators—some focusing on decoding accuracy, others on decoding efficiency—corresponding weighting coefficients need to be introduced for each indicator. Finally, the subject's EEG performance on different tasks is scored (e.g., ...). Figure 5 ):

[0106]

[0107]

[0108] Where K is the reference standard for decoding accuracy, a is the accuracy of task A, and T is the accuracy of the decoding. train The standard for representing training time, T test A reference standard for indicating test time. This represents the training time for task A. This indicates the test time for task A. This indicates the training time for task B. This represents the testing time for task B, with different indicator weight coefficients X1 + X2 + X3 = 1. After calculating the evaluation score, we can then make an appropriate task decision output based on the evaluation score:

[0109]

[0110] When S A >S B When S indicates that task A has a higher weight score, the appropriate decision result for the task is A. B >S A When this occurs, it indicates that task B has a higher weight score, and the appropriate decision outcome for task B is B. If S A <0 and S B If the value is less than 0, then recalibration is required.

[0111] In this embodiment, before obtaining EEG data, conductive gel is injected between the electrodes and the scalp to keep the electrode impedance below 10kΩ. The motor imagery task involves visual cues for "left hand" and "right hand" motor imagery. In the verbal imagery task, two words ("go" and "stop") are visually cued, and subjects perform the imagining of phonation or whispering of these two words. Considering commonly used public dataset settings and subsequent cross-task decoding network data input, the motor imagery execution cycle is set to 4 seconds. Compared to motor imagery, which can be continuously and repeatedly performed, the verbal imagery task only requires one to two seconds to imagine word pronunciation; therefore, the verbal imagery cycle is set to 2 seconds and repeated 4 times. The number of subjects for multi-task EEG data collection is planned to be 10, with participants being healthy graduate students aged 22-30 years.

[0112] To verify the decoding performance of the proposed method on motor imagery EEG data, some parameters in the model training were analyzed and adjusted. The entire experimental platform was based on Python 3.8, using the deep learning toolbox PyTorch and the EEG analysis toolbox MME.

[0113] This invention transforms collected EEG data into three different perspectives—spatial connectivity, spatiotemporal, and time-frequency—as feature inputs. A convolutional neural network is constructed, with the input data being the multi-perspective EEG features processed in the previous step. The network uses the ReLU activation function and a softmax output layer for nonlinear transformation and multi-classification. During training, the Adam optimizer and cross-entropy loss function are used to optimize network weights and biases, ensuring training effectiveness and high recognition accuracy. The trained network model is then used to evaluate its performance on independent test datasets. EEG data from motor imagery and verbal imagery tasks are analyzed, and a comprehensive evaluation is performed based on multiple performance metrics, including decoding accuracy, training time, and testing time. By comparing these metrics, a score for each task type is calculated, and weight coefficients are applied to adjust the influence of each metric. Based on the comprehensive score, the most suitable task for the test subject is determined. If task A scores higher than task B, task A is recommended; otherwise, task B is recommended. If the scores for the two tasks are equal, additional testing or parameter adjustments may be necessary to make a final decision.

[0114] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A task assessment method applicable to individual test subjects, characterized in that, Includes the following steps: (1) Construct different experimental paradigms based on different tasks and extract EEG data for different tasks; (2) By using a multi-view feature extraction network, EEG data is converted into different perspectives of EEG and key EEG features related to the task are extracted. (3) Construct a deep learning network model, extract features of different types of time-frequency maps, and decode the EEG features; (4) For the above deep learning network models, EEG data is used to evaluate the cross-task analysis performance of the models; Step (2) specifically includes: 1) Constructing a spatial connectivity perspective Calculate the correlation between signals from different brain regions: （2）； in, It is the first and the The correlation coefficient between the electrodes and They are at the time points electrodes and voltage value, and This is the corresponding average voltage value. This represents the total number of points in time. The calculated connectivity value This forms a connectivity matrix, where each element of the matrix represents the connection strength between a pair of electrodes; 2) Constructing a spatiotemporal perspective a Time Series Analysis Time-domain features were extracted using the mean and variance of EEG data. （3）； （4）； in, It is the first Each electrode at time point The input signal value, Is calculating the first The average voltage of each electrode over a given time period Indicates the method used to calculate the first... The variance of the individual electrode signals; b. Spatial Pattern Analysis Spatial filtering analysis was used to evaluate the distribution of EEG signals among different electrodes on the scalp. （5）； in, It is the first one after Laplace filtering The signal value of each electrode It is the original signal value. Indicates the first The adjacent electrodes of each electrode; 3) Constructing a time-frequency perspective Short-time Fourier transform is used to evaluate the local frequency characteristics of a signal: （6）； in, It is the first The signal from each electrode It is a window function used to localize the signal at time point t; Step (4) decodes and analyzes the cross-task EEG of a single subject based on a multi-view feature extraction network and a deep learning network model, determines the applicable task type for the single subject based on the analysis results, and scores the subject's EEG performance on different tasks: （12）； （13）； Where K is the reference standard for decoding accuracy, and a is the accuracy of task A. Standards for representing training time A reference standard for indicating test time. This represents the training time for task A. This indicates the test time for task A. This indicates the training time for task B. This represents the test time for task B, with different weighting coefficients for each metric. After calculating the evaluation score, the appropriate task decision output is made based on the evaluation score: （14）； when When the decision result for the applicable task is A, When the decision result for the applicable task is B, if If so, recalibrate.

2. The applicable task assessment method for individual subjects according to claim 1, characterized in that, In step (1), the sampling rate is set to 500 Hz, and a 60Hz notch filter is used to eliminate interference, as shown in the following formula: （1）； in, The notch frequency, The notch filter factor is 1. 2 is the notch filter factor, and S is the input signal frequency.

3. The applicable task assessment method for individual subjects according to claim 1, characterized in that, In step (3), a convolutional neural network consisting of 4 convolutional layers, 2 pooling layers, and 2 fully connected layers is constructed to extract features from time-frequency maps of different categories. Given n training samples, a convolutional layer receives input data, extracts sample features, and obtains a series of feature maps: （7）； In the above formula, As training samples, Indicates the size of the convolution kernel. This indicates the number of convolutional kernels, where conv(*) represents the defined convolutional layer. The ReLU activation function is used to make the output of each network layer non-linear: （8）； The input parameters, after being processed by convolution, activation, and pooling, are used to input feature maps into a classifier composed of fully connected layers for classification. Each fully connected layer consists of multiple neurons, with full connectivity between layers. The neurons in each layer are independent, and the expression for a single neuron is: （9）； in, Indicates the weight. To indicate the bias, the last layer in the fully connected layer is activated using the softmax function: （10）； In the above formula, Indicates the first The probability of a class This indicates that the category has been calculated. The network output is calculated using cross-entropy as the loss function, and the Adam optimizer is selected with an optimizer learning rate set to... The cross-entropy loss function is defined as follows: （11）； in, Indicates category The true label, Indicates the first The probability of a class It is the total number of training samples. It represents the number of categories.

Images