Cross-task mental load evaluation method based on electroencephalogram tensor transfer learning

By using tensor form to construct features and introducing transfer learning and discriminant ability alignment in the EEG cross-task evaluation, the problem of multi-dimensional feature neglect and insufficient discriminative characteristics in the existing technology is solved, and the accuracy of cross-task brain load evaluation is significantly improved. Rate.

CN118902472APending Publication Date: 2024-11-08INFORMATION SCI RES INST OF CETC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410937540.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing EEG cross-task brain load evaluation method ignores the multi-dimensional feature structure in feature processing, resulting in poor recognition accuracy and transfer learning does not consider feature discrimination, resulting in poor cross-task recognition effect.

Method used

The electroencephalogram features are constructed using tensor form, and the time-frequency features are aligned through transfer learning methods, feature discrimination ability is introduced, feature distribution distance is reduced under different tasks, and the accuracy of cross-task brain load evaluation is improved.

Benefits of technology

Through the tensor transfer learning method, the accuracy of cross-task brain load evaluation is significantly improved, especially in similar tasks and cross-task recognition effects, which improves the ability to discriminate brain load.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118902472A_ABST
    Figure CN118902472A_ABST
Patent Text Reader

Abstract

The disclosure provides a cross-task mental load evaluation method based on electroencephalogram tensor transfer learning, comprising: collecting electroencephalogram signals of a plurality of electroencephalogram channels under different tasks, each channel corresponding to different regions of the brain; calculating time-frequency characteristics of the electroencephalogram signals; constructing a target domain three-dimensional tensor containing the time-frequency characteristics of the multi-channel electroencephalogram signals; aligning the target domain three-dimensional tensor with the source domain three-dimensional tensor through a transfer learning method to obtain a three-dimensional tensor after transfer learning; performing feature screening on the three-dimensional tensor after transfer learning to obtain screened features; training a recognition model by using the screening features; and according to the identification model, identifying the mental load.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of brain-computer interface, and in particular to a cross-task mental workload evaluation method based on electroencephalogram (EEG) tensor transfer learning. Background Art

[0002] In human-computer interaction systems, the operator's mental workload status directly affects the effect of their task execution. Too high or too low mental workload levels will lead to a decline in the operator's task performance and even threaten their life safety. Accurate mental workload evaluation is of great significance to ensure the correct execution of tasks and the safety of operators. At present, commonly used mental workload evaluation methods include subjective evaluation method, task performance evaluation method and physiological evaluation method. Among them, physiological evaluation method has attracted the attention of a large number of researchers due to its objective, online and real-time characteristics.

[0003] Among many physiological indicators, EEG signals can directly reflect the cognitive processing of the human brain and are considered to be the most sensitive physiological indicator of mental workload. A large number of studies have confirmed that the linear characteristics (such as frequency band energy), nonlinear characteristics (such as sample entropy) and functional network characteristics of EEG signals can effectively distinguish the mental workload level of operators under specific tasks. However, when the same EEG features are used to evaluate the mental workload of different tasks (i.e., cross-task evaluation), the evaluation results are not ideal. Many studies attribute the cross-task problem of mental workload evaluation to the differences in brain information processing mechanisms when operators perform different tasks. Specifically, due to the different brain regions involved in the brain processing different information and the different response patterns of these brain regions, the EEG indicators under different tasks change with mental workload differently. From the statistical characteristics of the features, the EEG features under different tasks often show differences in marginal and conditional probability distributions, so it is impossible to achieve a relatively ideal recognition effect in classification tasks.

[0004] At present, EEG-based cross-task mental workload evaluation research is mainly divided into two categories, namely, common feature selection method and feature transfer learning method. The common feature selection method selects physiological indicators that change significantly with the change of mental workload in different tasks to obtain "common" indicators suitable for multiple tasks. These commonly used feature screening methods include statistical test methods, feature ranking methods, etc.

[0005] The method based on feature transfer learning aims to construct a feature transformation function / matrix that satisfies the objective function, so that the signal features under different tasks have the same probability distribution after transformation, and the transformed features are used for cross-task mental workload recognition, thereby achieving a higher recognition accuracy.

[0006] It is worth noting that the common feature selection method often loses the features that are effective for identifying the mental workload under a specific task, resulting in a decrease in accuracy. Existing transfer learning methods only consider the alignment of the marginal probability distribution or conditional probability distribution of features, but do not consider the discriminability of the features, so the recognition accuracy is poor.

[0007] The above two methods only consider the single-dimensional features of EEG (such as time, frequency, etc.) when extracting and constructing EEG features, and process them in the form of one-dimensional vectors when screening features or transferring learning. Since the brain's information processing is a dynamic, multi-brain region collaborative process, this feature processing method ignores the structural characteristics of multiple EEG feature dimensions such as different time, channels, and frequencies, resulting in the loss of useful information for discriminating mental workload.

[0008] To solve the above problems, a cross-task mental workload evaluation method based on EEG tensor transfer learning is proposed here. Summary of the invention

[0009] The embodiments of the present disclosure aim to solve at least one of the technical problems existing in the prior art, and provide a cross-task mental workload evaluation method based on EEG tensor transfer learning, which constructs EEG features in tensor form to represent the dynamic response characteristics of the brain in a task state, thereby obtaining more effective discriminant features, and then performs transfer learning on the EEG tensors under different tasks according to the feature distribution characteristics, which can effectively reduce the distance between the EEG feature distributions under different tasks, and at the same time introduces consideration of feature discrimination ability when aligning feature distributions, thereby improving the accuracy of cross-task mental workload evaluation.

[0010] In a first aspect of the present disclosure, a method for evaluating cross-task mental workload based on EEG tensor transfer learning is provided, the evaluation method comprising:

[0011] Collecting EEG signals from multiple EEG channels under different tasks, each channel corresponding to a different area of ​​the brain;

[0012] Calculating the time-frequency characteristics of the EEG signal;

[0013] Constructing a target domain three-dimensional tensor containing the time-frequency features of the EEG signal;

[0014] Aligning the target domain three-dimensional tensor with the source domain three-dimensional tensor through a transfer learning method to obtain a three-dimensional tensor after transfer learning;

[0015] Performing feature screening on the three-dimensional tensor after the transfer learning to obtain screening features;

[0016] Using the screening features to train a recognition model;

[0017] The mental workload is identified according to the identification model.

[0018] Furthermore, the tasks include a character memory task, an object memory task, a character space memory task and an object space memory task. The character space memory task includes memorizing the character space placement position, and the object space memory task includes memorizing the object space placement position.

[0019] Furthermore, the time-frequency characteristics of the EEG signal are calculated by the following formula:

[0020]

[0021] Among them, X k (c,f,t,k) represents the time-frequency distribution, k is the data segment, c is the EEG channel, f is the frequency value, t is the time, and n is the number of data segments.

[0022] Furthermore, the target domain three-dimensional tensor is aligned with the source domain three-dimensional tensor by a transfer learning method to obtain a transferred three-dimensional tensor, including:

[0023] Substitute the target domain and source domain three-dimensional tensor into the loss function L = L distribution+ θL discrimination ;

[0024] Among them, L distribution is the loss function for feature distribution alignment, L discrimination is the class discrimination loss function, and θ is the tuning factor.

[0025] Furthermore, the loss function L of the feature distribution alignment is distribution Calculated by the following formula:

[0026]

[0027] Where μ is the tuning factor, μ∈[0,1], L marginal For edge distribution alignment, L conditional Alignment for conditional distributions;

[0028] X S and X T are the sample tensors of the source domain and the target domain respectively, N S and N T are the number of samples, P is the source domain feature transformation matrix, U is the subspace projection matrix, C is the number of categories, and Represents the number of samples belonging to category C in the source domain and target domain samples.

[0029] Furthermore, the method further includes aligning the feature distribution with a loss function L distribution Expand according to mode k to obtain the loss function L that optimizes feature distribution alignment distribution :

[0030] L distribution =(1-μ)tr(U (k) X (k( Q 0 X (k)T U (k)T )+μtr(U (k) X (k) Q 1 X (k)T U (k)T )

[0031] Where tr(·) represents the trace of the matrix, k is the order of the eigentensor, Q 0 and Q 1 The MMD matrices corresponding to the edge and conditional probability alignment respectively;

[0032]

[0033] Furthermore, the class discrimination loss function L discrimination Calculated by the following formula:

[0034]

[0035] Among them, L between and L within Represent the inter-class interval and intra-class interval respectively, M S and M T are the means of source domain and target domain samples, respectively. and are the mean values ​​of samples belonging to category C in the source domain and the target domain, G S , G T is the feature tensor after projection mapping,

[0036] Furthermore, the method further includes converting the class discrimination loss function L discrimination Expand according to mode k to obtain the optimized class discrimination loss function L discrimination :

[0037] L discrimination =tr(U (k) (B (k) -ζW (k) ) (k)T )

[0038] Among them, B (k) and W(k) are the inter-class distance matrix and intra-class distance matrix under the k-order tensor, respectively, and ζ is a factor;

[0039]

[0040] mat (k) (·) is the mode k expansion of the tensor, and the factor ζ is taken as (W (k )) -1 ·B (k) The maximum eigenvalue of .

[0041] Furthermore, the method further comprises: distribution And the optimized class discrimination loss function L discrimination Substitute L = L distribution +θL discrimination Get the optimized loss function L:

[0042]

[0043] Furthermore, the method also includes fixing the source domain feature transformation matrix P to obtain the subspace projection matrix U or fixing the subspace projection matrix U to obtain the source domain feature transformation matrix P.

[0044] Furthermore, the three-dimensional tensor after the transfer learning is subjected to feature screening to obtain screening features, including:

[0045] For feature X i ,make and They are the features X in category c respectively. i The mean and standard deviation of i and σ i Substitute the following formulas for the mean and standard deviation of all samples:

[0046]

[0047] After sorting the FisherScore values ​​of each feature in descending order, the features corresponding to the first m FisherScore values ​​are taken as the screening features, where m is the preset value range.

[0048] Furthermore, the identifying of mental workload according to the identification model includes:

[0049] Recognition is performed in the character memory task, the object memory task, the character space memory task, and the object space memory task respectively, and verification is performed by 5-fold cross-check;

[0050] Any one of the character memory task, the object memory task, the character space memory task and the object space memory task is selected as a training task for training, and then verified through other tasks other than the training task. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A schematic diagram of a process of a cross-task mental workload evaluation method based on EEG tensor transfer learning according to an embodiment of the present disclosure;

[0052] Figure 2 It is a classification and recognition effect diagram of the same type of tasks and cross-type tasks in the embodiment of the present disclosure;

[0053] Figure 3 A comparison chart of the results of mental workload identification using the classical feature extraction method and the present method in the embodiment of the present disclosure;

[0054] Figure 4 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0055] Here, exemplary embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the embodiments of the present disclosure.

[0056] The terms used in the disclosed embodiments are only for the purpose of describing specific embodiments and are not intended to limit the disclosed embodiments. The singular forms of "a", "said" and "the" used in the disclosed embodiments and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0057] It should be understood that although the terms first, second, third, etc. may be used to describe various information in the disclosed embodiments, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the disclosed embodiments, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0058] like Figure 1 , which is a flow chart of a cross-task mental workload evaluation method based on EEG tensor transfer learning according to an embodiment of the present disclosure. The extraction method 100 comprises the following steps:

[0059] S101: Collecting EEG signals of multiple EEG channels under different tasks, each channel corresponding to a different area of ​​the brain;

[0060] S102: Calculating the time-frequency characteristics of the EEG signal;

[0061] S103: constructing a target domain three-dimensional tensor containing the time-frequency features of the EEG signal;

[0062] S104: aligning the target domain three-dimensional tensor with the source domain three-dimensional tensor through a transfer learning method to obtain a three-dimensional tensor after transfer learning;

[0063] S105: performing feature screening on the three-dimensional tensor after the transfer learning to obtain screening features;

[0064] S106: using the screening features to train a recognition model;

[0065] S107: Identify the mental workload according to the identification model.

[0066] The tasks included working memory tasks of four different information types (character, object, space (character), space (object)), and the task difficulty was divided into three levels (N ranged from 1 to 3), that is, all subjects needed to complete a total of 12 tasks (4 task types × 3 task difficulties) of mental workload.

[0067] Specifically, under each task condition, the subject needs to judge whether the currently presented information is consistent with the Nth previous information.

[0068] Each task condition contained 43 trials (the first three trials were not used for subsequent analysis). Each trial lasted 2.5 s, of which the information (stimulus) was presented for 0.5 s, and the subjects had to make a judgment within these 2.5 s.

[0069] The subjects' reaction time was recorded during the task, and the reaction accuracy was calculated after the task was completed. The subjects were also asked to fill in the NASA-TLX Mental Workload Subjective Evaluation Scale.

[0070] It should be noted that the task type and difficulty can be set by those skilled in the art according to actual needs, and this embodiment does not impose any specific restrictions.

[0071] Furthermore, the acquisition device used NeuroScan EEG acquisition system (SynAmps2, USA) to record the subjects' 60-lead EEG signals in real time, with a signal sampling rate of 1000 Hz. During the acquisition process, the electrode impedance was kept below 10 kΩ.

[0072] Preferably, the subjects are 8 males and 8 females with an average age of 25.6±2.4 years old.

[0073] Optionally, the collected EEG signal is also subjected to bandpass filtering (0.5-40 Hz) to eliminate the signal DC component and high-frequency noise.

[0074] Furthermore, the SASICA artifact recognition algorithm was used to remove artifacts such as blinking, vertical / horizontal electrooculography, and electromyography from the collected EEG signals after resampling at 256 Hz, and the signals (2.5 s) from 0.5 s before to 2 s after the stimulus presentation of each channel were taken for transfer learning.

[0075] The evaluation method is described in detail below.

[0076] The time-frequency characteristics of the EEG signal are calculated using the following formula:

[0077]

[0078] Among them, X k (c,f,t,k) represents the time-frequency distribution, k is the data segment, c is the EEG channel, f is the frequency value, t is the time, and n is the number of data segments.

[0079] Time-frequency distribution is a method of expressing how a signal changes in time and frequency. k (c, f, t, k) represents the time-frequency distribution value of the EEG signal under a specific channel, frequency, time and data segment, representing the energy of the signal under that condition. By calculating the average of these values ​​(ie, ERSP), the average energy distribution of the EEG signal under different conditions is obtained, thereby further analyzing the activity state of the brain.

[0080] Construct a three-dimensional tensor of the target domain containing the time-frequency features of the EEG signal. In this three-dimensional tensor of the target domain:

[0081] Channel (c): As the first dimension, it represents different EEG electrodes or channels. Each channel will have an associated two-dimensional array (matrix) that describes the average energy at different frequencies and time points under that channel.

[0082] Frequency (f) and time (t): These two dimensions together constitute the rows and columns of the above two-dimensional array (matrix). Specifically, frequency can be regarded as the rows of the matrix and time can be regarded as the columns of the matrix (or vice versa).

[0083] Average Energy (ERSP): The actual data value stored in the 3D tensor. For each (channel, frequency, time) combination, a corresponding average energy value is stored in the 3D tensor.

[0084] Among them, the target domain three-dimensional tensor is aligned with the source domain three-dimensional tensor through the transfer learning method, and the three-dimensional tensor obtained after transfer learning includes:

[0085] Substitute the target domain three-dimensional tensor into the loss function L = L distribution +θL discrimination ;

[0086] Among them, L distribution is the loss function for feature distribution alignment, L discrimination is the class discrimination loss function, and θ is the tuning factor.

[0087] Furthermore, when θ = 0, the loss function degenerates into a tensorized joint domain alignment method.

[0088] Among them, feature distribution alignment assumes that the source domain and target domain features are in the subspace U = {U (k)} k=1,...,K The projections on the source domain have similar distributions. Considering that the feature distributions of different tasks vary greatly, we introduce the source domain feature transformation matrix P = {P (k)} k=1,...,K .

[0089] Among them, the loss function of feature distribution alignment consists of two parts: edge alignment and conditional distribution alignment, expressed as L distribution =(1-μ)L marginal +μL conditional ;

[0090] Among them, (μ is the tuning factor, μ∈[0,1]), the distribution distance is measured using the MMD criterion.

[0091] The calculation formula for distribution alignment is as follows:

[0092]

[0093] Among them, X S and X T They are sample tensors of the source domain and the target domain (the number of samples is N S and N T ), which correspond to the features of different tasks in the present invention; P is the source domain feature transformation matrix, U is the subspace projection matrix, Represents X S Modular multiplication with P, that is C is the number of categories, and Represents the number of samples belonging to category C in the source domain and target domain samples.

[0094] Preferably, the loss function L that aligns the feature distribution is distribution Expand according to the modulus k to obtain the loss function L that optimizes the feature distribution alignmentdistribution :

[0095] L distribution =(1-μ)tr(U (k) X (k) Q 0 X (k)T U (k)T )+μtr(U (k) X (k) Q 1 X (k)T U (k)T )

[0096] Where tr(·) represents the trace of the matrix, k is the order of the eigentensor, Q 0 and Q 1 The MMD matrices corresponding to the edge and conditional probability alignment respectively;

[0097]

[0098]

[0099] Among them, this type of discriminant loss function L discrimination Calculated by the following formula:

[0100]

[0101] Among them, L between and L within Represent the inter-class interval and intra-class interval respectively, M S and M T are the means of source domain and target domain samples, respectively. and are the mean values ​​of samples belonging to category C in the source domain and the target domain, G S , G T is the feature tensor after projection mapping,

[0102] Preferably, this class of discriminant loss function L discrimination According to the module k expansion, we can get the optimized class discrimination loss function L discrimination :

[0103] L discrimination =tr(U (k) (B (k) -ζW (k) ) (k)T )

[0104] Among them, B (k) and W (k) are the inter-class distance matrix and intra-class distance matrix under the k-order tensor, respectively, and ζ is a factor;

[0105]

[0106] mat (k) (·) is the mode k expansion of the tensor, and the factor ζ is taken as (W (k) ) -1 ·B (k) The maximum eigenvalue of .

[0107] Preferably, the method further comprises a loss function L for aligning the optimized feature distribution distribution And the optimized class discrimination loss function L discrimination Substitute L = L distribution +θL discrimination Get the optimized loss function L:

[0108]

[0109] Since the objective function has multiple items to be optimized (U and P), an alternating iterative strategy is used to solve it. The specific steps are as follows:

[0110] Fix P and solve U: Solve U through the high-order singular value decomposition (HOSVD) of the tensor and introduce auxiliary variables The optimization problem (8) can be transformed into The singular value decomposition problem.

[0111] Fix U and solve P: After fixing U, the only term related to P is L distribution , that is, the optimization goal is as follows:

[0112]

[0113] After expanding the above formula according to mode k, it can be converted into an optimization problem shown in the following formula, which can be solved by classical optimization methods.

[0114]

[0115] Through the Zhang subspace U and the alignment matrix P, we can finally get the feature representation of the source domain and the target domain in the Zhang subspace and As shown in the following formula:

[0116]

[0117] The three-dimensional tensor after transfer learning is subjected to feature screening, and the screening features include:

[0118] For feature X i ,make and They are the features X in category c respectively. iThe mean and standard deviation of i and σ i Substitute the following formulas for the mean and standard deviation of all samples:

[0119]

[0120] After sorting the FisherScore values ​​of each feature in descending order, the features corresponding to the first m FisherScore values ​​are taken as the screening features, where m is the preset value range.

[0121] The recognition model includes the same type of task recognition and cross-type task recognition;

[0122] The same type of task recognition includes recognition in the character memory task, the object memory task, the character space memory task and the object space memory task respectively and verification by 5-fold cross-validation;

[0123] 5-fold cross validation (also known as 5-fold cross validation) is a commonly used method for evaluating machine learning models, especially when dealing with limited data sets. This method evaluates model performance by dividing the dataset into five equal parts (or "folds"), and then using four of them as training sets and the remaining one as a validation set in a specific order, performing five independent training and validation processes to obtain an evaluation of model performance;

[0124] Specifically, first, the original dataset is divided into five subsets (or “folds”) of equal size.

[0125] Secondly, training and verification:

[0126] First time: select the first fold as the validation set, and the remaining four folds (the second, third, fourth, and fifth folds) as the training set. Train the model and evaluate it on the validation set.

[0127] Second time: select the second fold as the validation set, and the remaining four folds (the first, third, fourth, and fifth folds) as the training set. Train the model again and evaluate it on the new validation set.

[0128] ...(repeat the above process)

[0129] Fifth time: select the 5th fold as the validation set, and the remaining 4 folds (1st, 2nd, 3rd, and 4th) as the training set for the final training and validation.

[0130] Finally, perform performance evaluation: record the model performance (such as accuracy, recall, F1 score, etc.) of each validation, and calculate the average of these performance indicators as the final performance evaluation of the model under 5-fold cross validation.

[0131] The cross-category task recognition includes selecting any one of the character memory task, the object memory task, the character space memory task and the object space memory task as a training task for training, and then verifying through other tasks outside the training task.

[0132] Specifically, a three-classification recognition model was constructed using a support vector machine classifier for both intra-task and cross-task conditions. For intra-task mental workload recognition, a 5-fold cross-validation method was used for each subject; for cross-task mental workload recognition, one of the four tasks was selected as a training task, and the other three were selected as test tasks for verification. In order to evaluate the recognition effect, indicators such as accuracy, sensitivity, and specificity were calculated.

[0133] Figure 2 The classification and recognition effects of the same type of tasks and cross-type tasks in this embodiment are shown. As can be seen from the table, the average recognition accuracy of the proposed EEG feature tensor representation method in four tasks can reach 92.7%, and the average recognition accuracy of mental workload under multiple cross-tasks is 81.3%.

[0134] Figure 3 The results of mental workload identification using classical feature extraction methods are compared with the method of the present invention. These classical signal features include band energy, multi-scale sample entropy, and EEG time-frequency-space tensor features without transfer learning. As can be seen from the figure, although the feature extraction method based on time domain (multi-scale sample entropy) and frequency domain (band energy) can achieve relatively ideal results in mental workload identification of similar tasks (sample entropy: 0.662±0.016, band energy: 0.685±0.024, sample entropy + band energy: 0.736±0.025), its effect in cross-class task situations is not ideal (sample entropy: 0.394±0.018, band energy: 0.399±0.020, sample entropy + band energy: 0.408±0.025).

[0135] Compared with these single-dimensional features, the recognition accuracy of tensor features without transfer learning is significantly improved in both similar and cross-category tasks (same-category tasks: 0.871±0.079, cross-category tasks: 0.625±0.029). The feature extraction method based on tensor representation and transfer learning proposed in the present invention has the highest recognition accuracy.

[0136] The embodiments of the present disclosure provide a cross-task mental workload evaluation method based on EEG tensor transfer learning, which constructs EEG features in tensor form to represent the dynamic response characteristics of the brain in a task state, thereby obtaining more effective discriminant features, and then performs transfer learning on the EEG tensors under different tasks according to the feature distribution characteristics, which can effectively reduce the distance between EEG feature distributions under different tasks, and at the same time introduce consideration of feature discrimination ability when aligning feature distributions, thereby improving the accuracy of cross-task mental workload evaluation.

[0137] The electronic device 400 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 400 may include, but is not limited to, a processor 401 and a memory 402. Those skilled in the art will appreciate that Figure 4 It is only an example of the electronic device 400 and does not constitute a limitation of the electronic device 400. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0138] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0139] The memory 402 may be an internal storage unit of the electronic device 400, for example, a hard disk or memory of the electronic device 400. The memory 402 may also be an external storage device of the electronic device 400, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 400. Further, the memory 402 may also include both an internal storage unit of the electronic device 400 and an external storage device. The memory 402 is used to store the computer program 403 and other programs and data required by the electronic device. The memory 402 may also be used to temporarily store data that has been output or is to be output.

[0140] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present disclosure. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0141] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0142] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0143] In the embodiments provided in the present disclosure, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection of devices or units, which may be electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0146] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present disclosure implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0147] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be included in the protection scope of the present disclosure.

Claims

1. A cross-task mental workload evaluation method based on EEG tensor transfer learning, characterized in that: The evaluation method includes: Collecting EEG signals from multiple EEG channels under different tasks, each channel corresponding to a different area of ​​the brain; Calculating the time-frequency characteristics of the EEG signal; Constructing a target domain three-dimensional tensor containing the time-frequency features of the EEG signal; Aligning the target domain three-dimensional tensor with the source domain three-dimensional tensor through a transfer learning method to obtain a three-dimensional tensor after transfer learning; Performing feature screening on the three-dimensional tensor after the transfer learning to obtain screening features; Using the screening features to train a recognition model; The mental workload is identified according to the identification model.

2. The method according to claim 1, characterized in that: The tasks include a character memory task, an object memory task, a character space memory task and an object space memory task. The character space memory task includes memorizing the character space placement position, and the object space memory task includes memorizing the object space placement position.

3. The method according to claim 1, characterized in that The time-frequency characteristics of the EEG signal are calculated by the following formula: Among them, X k (c,f,t,k) represents the time-frequency distribution, k is the data segment, c is the EEG channel, f is the frequency value, t is the time, and n is the number of data segments.

4. The method according to claim 1, characterized in that: The target domain three-dimensional tensor is aligned with the source domain three-dimensional tensor by a transfer learning method to obtain a transferred three-dimensional tensor, including: Substitute the target domain and source domain three-dimensional tensor into the loss function L = L distribution +θL discrimination ; Among them, L distribution is the loss function for feature distribution alignment, L discrimination is the class discrimination loss function, and θ is the tuning factor.

5. The method according to claim 4, characterized in that The loss function L of the feature distribution alignment distribution Calculated by the following formula: Where, is the tuning factor, μ∈[0,1], L marginal For edge distribution alignment, L conditional Alignment for conditional distributions; X S and X T are the sample tensors of the source domain and the target domain respectively, N S and N T are the number of samples, P is the source domain feature transformation matrix, U is the subspace projection matrix, C is the number of categories, and Represents the number of samples belonging to category C in the source domain and target domain samples.

6. The method according to claim 5, characterized in that The method also includes aligning the feature distributions with a loss function L distribution Expand according to mode k to obtain the loss function L that optimizes feature distribution alignment distribution : L distribution =(1-μ)tr(U (k) X (k) Q0X (k)T HER (k)T )+µtr(U (k) X (k) Q1X (k)T HER (k)T ) Where tr(·) represents the trace of the matrix, k is the order of the eigentensor, Q0 and Q1 correspond to the MMD matrices of edge and conditional probability alignment, respectively; 7. The method according to claim 1, characterized in that The class discrimination loss function L discrimination Calculated by the following formula: Among them, L between and L within Represent the inter-class interval and intra-class interval respectively, M S and M T are the means of source domain and target domain samples, respectively. and are the mean values ​​of samples belonging to category C in the source domain and the target domain, G S , G T is the feature tensor after projection mapping, 8. The method according to claim 7, characterized in that The method also includes converting the class discrimination loss function L discrimination Expand according to mode k to obtain the optimized class discrimination loss function L discrimination : L discrimination =tr(U (k) (B (k) -ζW (k) )U (k)T ) Among them, B (k) and W (k) are the inter-class distance matrix and intra-class distance matrix under the k-order tensor, respectively, and ζ is a factor; mat (k) (·) is the mode k expansion of the tensor, and the factor ζ is taken as (W (k) ) -1 ·B (k) The maximum eigenvalue of .

9. The method according to claim 6 or 8, characterized in that: The method further comprises aligning the optimized feature distribution with a loss function L distribution And the optimized class discrimination loss function L discrimination Substitute L = L tribution +θL discrimination Get the optimized loss function L:

10. The method according to claim 9, characterized in that The method also includes fixing the source domain feature transformation matrix P to obtain the subspace projection matrix U or fixing the subspace projection matrix U to obtain the source domain feature transformation matrix P.

11. The method according to claim 1, characterized in that: The performing feature screening on the three-dimensional tensor after the transfer learning to obtain the screening features includes: For feature X i ,make and They are the features X in category c respectively. i The mean and standard deviation of i and σ i Substitute the following formulas for the mean and standard deviation of all samples: After sorting the FisherScore values ​​of each feature in descending order, the features corresponding to the first m FisherScore values ​​are taken as the screening features, where m is the preset value range.

12. The method according to claim 1, characterized in that The identifying of the mental workload according to the identification model includes: Recognition is performed in the character memory task, the object memory task, the character space memory task, and the object space memory task respectively, and verification is performed by 5-fold cross-check; Any one of the character memory task, the object memory task, the character space memory task and the object space memory task is selected as a training task for training, and then verified through other tasks other than the training task.