Method and system for attention state evaluation based on brain-computer interface and multi-modal signals

Abstract

The application discloses a kind of based on brain-computer interface and multimodal signal's attention state evaluation method and system.The attention state evaluation method includes composition stage, feature extraction stage, feature learning stage and attention evaluation stage, it is characterized in that, composition stage includes based on the dominant contribution of different brain regions to different tasks Construction brain function area dynamic diagram, And with different brain function area as trigger, physiological signal is the multi-modal signal function connection dynamic diagram of reaction;Steps such as constructing multi-region dynamic diagram convolution network for attention state task.The application extracts the multimodal signal collaborative paradigm related to target attention task, with strong correlation characteristics as classification basis, to provide the contribution factor of different functional area electroencephalogram to target attention task, and the influence relationship of the correlation relationship between multimodal signals to target attention task, improve the reliability and stability of brain-computer interface attention evaluation.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, and more specifically, to a method and system for assessing attentional state based on brain-computer interface and multimodal signals. Background Technology

[0002] Brain-computer interface (BCI) is a middleware that establishes a direct connection between the human brain and external devices such as computers. It translates abstract and complex brain signals into machine-understandable commands, thereby enabling information exchange between the brain and the device. BCI has a wide range of applications, including medical rehabilitation, gaming and virtual reality, and human-computer control. Currently, most applications of BCI focus on the analysis and deconstruction of brain functional areas and human behavior.

[0003] Brain cognition and other studies indicate that a person's attention is controlled collaboratively by different functional areas in the brain. For example, the prefrontal cortex, in particular, plays a role in guiding and maintaining attention. In addition, this area can also suppress distractions. The superior parietal lobe is mainly responsible for spatial attention, while the inferior parietal lobe guides a person's attention to specific stimuli.

[0004] However, existing multimodal brain-computer interfaces, especially those for attention-related tasks, often perform feature-level fusion of extracted features from different signals or decision-level fusion of outputs from different modalities in analyzing multimodal signals. This neglects the corresponding functions of different brain regions and the strong correlation between these regions in driving human behavior. Furthermore, feature extraction that doesn't consider the relationship between different brain regions and corresponding tasks often results in poor interpretability, hindering further research into human cognition and subsequent advancements. Summary of the Invention

[0005] To address the shortcomings of existing technologies that neglect the correlation between different brain functional areas and target tasks, this invention provides a method and system for assessing attentional states based on brain-computer interfaces and multimodal signals. It constructs a multi-region dynamic graph convolutional network based on the influence of different brain functional areas on different attention-related sub-tasks, and on the correlation between different collaborative response patterns of EEG signals and other physiological signals and different attention-related sub-tasks. It extracts collaborative paradigms of multimodal signals with strong correlation characteristics related to the target attentional task as a classification basis, thereby providing the contribution factors of EEG signals from different functional areas to the target attentional task, and the influence relationship of the correlation between multimodal signals on the target attentional task, enabling a refined assessment of attentional states.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for assessing attentional states using a brain-computer interface, comprising a signal acquisition stage, a feature extraction stage, a feature learning stage, and an attentional state assessment stage, characterized in that it further comprises a mapping stage; the mapping stage includes the following steps:

[0007] Step S101: Design different attention evaluation subtask paradigms;

[0008] Step S102: Construct a dynamic map of brain functional regions based on the dominant contributions of different brain regions to different tasks, and establish a feature paradigm for the target task based on EEG signals; construct a dynamic map of multimodal signal functional connectivity with different brain functional regions as triggers and other physiological signals as responses, to obtain the collaborative paradigm established by each EEG functional region and the response physiological signals for the target attention sub-task, the relationship between multimodal signals and attention tasks regarding action intentions and response behaviors, and the response mechanism from intention to behavior; combine the dynamic map of brain functional regions and the dynamic map of multimodal signal functional connectivity to construct a multi-region dynamic map convolutional network for attention state tasks;

[0009] The feature extraction stage uses the multi-region dynamic graph convolutional network to extract relevant features from different experimental data of different target attention subtasks.

[0010] The feature learning stage trains the model to learn the standard paradigm of different attentional state subtasks by training multiple subjects with features of subtasks under different attentional states.

[0011] The attention assessment phase collects multimodal signals from individuals undergoing attention-related task training or rehabilitation via brain-computer interface. The standard paradigms for attention-related tasks learned through the brain functional area dynamic map and the multimodal signal functional connectivity dynamic map provide assessment criteria and effective guidance for patients.

[0012] Furthermore, the attention assessment subtask paradigm in step S101 assesses the subject's attentional ability from five dimensions: attention selection, attention retention, attention allocation, attention control, and attention span.

[0013] Furthermore, in step S102, the dynamic brain functional area map assembles all the main leads in the prefrontal cortex of the brain into a main functional area, and other auxiliary functional areas are correlated with the main functional area according to their relative positions and the degree of auxiliary contribution to different target sub-tasks, that is, the graph adjacency relation matrix is ​​initialized.

[0014] Furthermore, in step S102, the other physiological signals include electrocardiogram, electrodermal signal, and electromyography (EMG) signals.

[0015] Secondly, the present invention provides a multimodal signal attention assessment and analysis system for brain-computer interfaces, characterized in that it implements the attention state assessment method described above, comprising:

[0016] The data acquisition module collects multimodal signals from subjects when performing attention-related tasks and constructs different attention assessment subtask paradigms.

[0017] The mapping module constructs dynamic maps of brain functional areas based on the dominant contributions of different brain regions to different tasks, and constructs dynamic maps of multimodal signal functional connectivity triggered by different brain functional areas and responded to by physiological signals.

[0018] The feature extraction module combines the dynamic maps of brain functional areas and the dynamic maps of multimodal signal functional connectivity to construct a multi-region dynamic map convolutional network for attentional state tasks, capture response paradigms, and extract relevant features.

[0019] The feature learning module learns and emphasizes the relevance characteristics related to the target task from the relevance features extracted by the multi-region dynamic graph convolutional network.

[0020] The attention evaluation module further integrates and reduces the dimensionality of features through several fully connected neural networks, and finally provides a comprehensive evaluation result for different user attention paradigms.

[0021] Furthermore, the dynamic map of brain functional areas is defined using the Pearson correlation coefficient; given input Where N single Let r represent the number of lead nodes in a single modality, and r represent the feature dimension of each lead signal. Then, the dynamic brain functional area map of this input is A. br Where for p,q∈{1,…,N single},A br-pq It can be calculated using the following formula:

[0022]

[0023] Furthermore, in the multimodal signal functional connection dynamic diagram, for multimodal signals of the same task, the multimodal signals are first feature-aligned through feature mapping, and then the multimodal signal input is given. Where N mm Let be the total number of multimodal signal lead nodes, and r be the aligned feature dimension of each lead signal. Then, the multimodal functional connection dynamic graph of this input is A. mm Where for p,q∈{1,…,N mm},A mm-pq It can be calculated using the following formula:

[0024]

[0025] Where, ω T ∈R F*1 For a learnable vector, ReLU(·) is the ReLU operation.

[0026] Furthermore, the feature learning module employs an Encoder framework based on a multi-head attention mechanism within the Transformer architecture to learn and emphasize the relevance characteristics related to the target task extracted by the multi-region dynamic graph convolutional network, resulting in an Encoder output Attention module composed of n self-attention modules. n The input V of the i-th self-attention module i V is the input to the (i-1)th self-attention module. i =Attention i-1 The output of the i-th self-attention module is Attention. i It can be calculated using the following formula:

[0027]

[0028] Among them, Q i ,K i Vi i Let be the feature vector of the query, key, and value in the i-th self-attention module, where sqrt(·) is the square root operation, and d o Indicates the dimension of the input features.

[0029] Compared with the prior art, the present invention has the following technical effects:

[0030] (1) The attention assessment method based on multimodal brain-computer interface of this invention focuses on the strength of the connection between different brain functional areas and different attention tasks, and emphasizes the different contributions of different brain functional areas to attention state tasks at different times.

[0031] (2) This invention considers other physiological signals besides EEG signals, and emphasizes the contribution of specific brain regions and other different physiological response signals to the synergistic paradigm of attention tasks, further improving the reliability and stability of attention assessment;

[0032] (3) This invention constructs a radiation synergy paradigm based on EEG, especially the main brain functional areas under different attention tasks, from brain to heart, from brain to skin and from brain to muscle, and thereby infers the standard paradigm of each physiological signal under specific attention tasks, thus providing more evidence-based guidance and feedback for attention training and rehabilitation.

[0033] (4) This invention involves not only the recognition and analysis of the attention state of an individual as a whole, but also the multi-region dynamic graph convolutional network established by this invention for each individual. It emphasizes that each graph focuses on the corresponding target sub-task, making the attention recognition and analysis more refined, so as to be able to adapt to different application scenarios, such as observing road conditions while driving, and detecting and training the attention-related ability qualifications of the target. Attached Figure Description

[0034] Figure 1 This is a flowchart of the attention state evaluation method in one embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but these are not intended to limit the scope of the invention.

[0036] In the following detailed description, numerous specific details are set forth to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that well-known algorithms and models are not shown in detail to avoid obscuring the spirit of the invention. The detailed description includes technical terms, such as attention paradigms referring to different attention tasks, which will be understood by those skilled in the art according to their conventional meaning.

[0037] Furthermore, the execution order of actions, steps, etc. in the apparatus and methods shown in the claims, specification, and drawings can be implemented in any order, unless a specific order is explicitly specified, and as long as the output of the preceding processing is not used in the subsequent processing.

[0038] Example 1

[0039] See Figure 1 This embodiment provides a brain-computer interface-based attention state assessment method, including a signal acquisition stage S2, a feature extraction stage S3, and an attention state assessment stage S4, as well as a mapping stage S1. The mapping stage S1 includes the following steps:

[0040] Step S101: Design different attention evaluation subtask paradigms;

[0041] Step S102: Construct a dynamic map of brain functional regions based on the dominant contributions of different brain regions to different tasks, and establish a feature paradigm for the target task based on EEG signals; construct a dynamic map of multimodal signal functional connectivity with different brain functional regions as triggers and other physiological signals as responses, to obtain the collaborative paradigm established by each EEG functional region and the response physiological signals for the target attention sub-task, the relationship between multimodal signals and attention tasks regarding action intentions and response behaviors, and the response mechanism from intention to behavior; combine the dynamic map of brain functional regions and the dynamic map of multimodal signal functional connectivity to construct a multi-region dynamic map convolutional network for attention state tasks;

[0042] In the feature extraction stage S2, the multi-region dynamic graph convolutional network is used to extract relevant features from different experimental data of different target attention subtasks.

[0043] The feature learning stage S3 trains the model to learn the standard paradigms of different attentional state subtasks by training on the features of multiple subjects in different attentional state subtasks.

[0044] During the attention assessment phase S4, multimodal signals from individuals undergoing attention-related task training or rehabilitation via brain-computer interface are collected. The standard paradigms for attention-related tasks learned through the dynamic maps of brain functional areas and the dynamic maps of multimodal signal functional connectivity are used to provide assessment criteria and effective guidance to patients.

[0045] The graph construction stage S1 can be before the feature extraction stage, where the relationships between nodes are defined in a predefined way, or it can be between the feature extraction stage and the feature learning stage, where the defined graph is applied to the extracted features and then these features are learned.

[0046] As a preferred technical solution, the attention assessment sub-task paradigm in step S101 evaluates the subject's attentional ability from five dimensions: attention selection, attention retention, attention allocation, attention control, and attention span. Furthermore, the attention assessment sub-task paradigm evaluates the subject's attentional ability from these five dimensions to consider the contribution of different brain regions to different attentional tasks, and the impact of collaborative work between different brain regions on the target task within the same attentional task.

[0047] Different brain region partitioning maps are constructed based on the dominant control roles of different brain functional areas for different sub-tasks. For example, the prefrontal cortex, in particular, plays a role in guiding and maintaining attention. Further, in step S102, the dynamic brain functional region map groups all major leads in the prefrontal cortex into a primary functional area. Other auxiliary functional areas are correlated with the primary functional area based on their relative positions and their degree of contribution to different target sub-tasks, i.e., an initial graph adjacency matrix is ​​formed. Thus, a multi-domain map based on the functional partitioning of different brain functional areas is constructed, establishing a feature paradigm for target tasks primarily based on EEG signals.

[0048] As a preferred technical solution, further: in step S102, the other physiological signals include electrocardiogram (ECG), electrodermal signaling (EDS), and electromyographic (EMG) signals, to construct a radiating and coordinated paradigm centered on the main brain functional areas under different attentional tasks, from brain to heart, from brain to skin, and from brain to muscle. This paradigm, in turn, infers the standard paradigm of each physiological signal under specific attentional tasks, thus providing more evidence-based guidance and feedback for attention training and rehabilitation. The standard paradigm here is reflected in the connection relationships between nodes in the graph network. For example, for task 1, the graph shows that nodes 1 and 2 are relatively active; for task 2, nodes 3 and 4 are relatively active; and nodes 5 and 6 show synchronization (or strong connectivity). This is the paradigm learned by the model. When an unfamiliar person performs task 1, the model finds that their nodes 1 and 2 are inactive, indicating that their ability to perform this task is far from the standard, or that they have a deficiency in performing task 1. Through guidance and training, they can gradually form a pattern where nodes 1 and 2 are relatively active, indicating that they have improved their ability in task 1.

[0049] Example 2

[0050] This embodiment provides a multimodal signal attention assessment and analysis system for brain-computer interfaces, implementing the attention state assessment method described in Embodiment 1, including:

[0051] The data acquisition module collects multimodal signals from subjects when performing attention-related tasks and constructs different attention assessment subtask paradigms.

[0052] The mapping module constructs dynamic maps of brain functional areas based on the dominant contributions of different brain regions to different tasks, and constructs dynamic maps of multimodal signal functional connectivity triggered by different brain functional areas and responded to by physiological signals.

[0053] The feature extraction module combines the dynamic maps of brain functional areas and the dynamic maps of multimodal signal functional connectivity to construct a multi-region dynamic map convolutional network for attentional state tasks, capture response paradigms, and extract relevant features.

[0054] The feature learning module learns and trains the model to learn the standard paradigms of different attentional state subtasks by learning and training the multi-domain graph representations of multiple subjects in different attentional state subtasks.

[0055] The attention assessment module further integrates and reduces the dimensionality of features through several fully connected neural networks, and finally provides a comprehensive assessment result of different attention paradigms of the subjects.

[0056] Examples of specific implementation methods for each of the above modules are as follows:

[0057] (1) Data acquisition module

[0058] First, different attention assessment paradigms were designed, including the attention allocation paradigm, attention selection paradigm, attention maintenance paradigm, attention span paradigm, and attention allocation paradigm, to evaluate the subjects' attentional abilities across five different dimensions. For example, in the attention span paradigm, subjects were shown an image containing a random number of black dots for 0.5 seconds, and then asked to memorize the number of black dots in the image by typing their answer. During this process, the subjects' physiological signals and behavioral data, such as reaction time and accuracy, were recorded. Feedback was provided to the subjects to help them adjust and complete the entire experimental procedure, which lasted 3-5 minutes. Because attentional paradigm tasks are necessary for data collection, the experimental paradigms were explained in advance during the data collection phase.

[0059] During attention-related tasks, subjects were guided to perform different instructions using various picture stimuli. Simultaneously, four modalities of electroencephalography (EEG), electrocardiography (ECG), electrodermal conductance (EDA), and electromyography (EMG) signals were recorded, along with behavioral data on attentional responses, such as reaction time and accuracy. This behavioral data can be used to construct average behavioral indicators for attention-related tasks in a large-sample setting, providing relatively objective evaluation indicators and guidance for rehabilitation and training in these tasks.

[0060] (2) Composition Module

[0061] To broaden the applicability of the backend multimodal attention state assessment model to attention-related tasks and to reduce subject dependence, the module for constructing multi-region dynamic maps in this example mainly consists of two parts: first, constructing dynamic maps of brain functional regions based on the dominant contributions of different brain regions to different tasks; and second, constructing dynamic maps of multimodal signal functional connectivity based on the functional connectivity correlations of multimodal signals. First, the collected signals corresponding to different modalities for different tasks are filtered, and then different dynamic maps are constructed.

[0062] 1) Construct dynamic brain function maps based on the dominant contributions of different brain regions to different tasks.

[0063] As mentioned in the technical background above, different brain regions dominate different attentional tasks. For example, areas like the brainstem and reticular activating system are involved in regulating arousal and its intensity. The brain regions corresponding to the brainstem can be represented by leads Fp1, Fp2, F7, F8, T7, and T8 on a 10-20 system-based EEG cap. More specifically, we can use these regions as the primary domains for attentional tasks requiring high-intensity alertness. These primary domains can then establish different combinations of connections with other brain regions ranked in terms of their correlation with the target task, in order of increasing strength. We can explore the paradigm characteristics of target tasks dominated by the primary domain and coordinated by multiple domains through dynamic graph networks. We use the Pearson Correlation Coefficient (PCC) to define redundant dynamic graphs of brain functional areas. Given input... Where N single Let r represent the number of lead nodes in a single modality, and r represent the feature dimension of each lead signal. Then the dynamic brain region map for this input is A. br Where for p, q∈{1,…N single},A br-pq It can be expressed by equation (1):

[0064]

[0065] 2) Constructing a dynamic graph of multimodal signal functional connectivity based on the correlation of multimodal signal functional connectivity.

[0066] To explore the response paradigms of various modal signals within the human body to different attentional tasks, this example establishes a multi-domain dynamic graph based on the correlation between responses of different brain regions and different physiological signals during the mapping phase. Similarly, for attentional tasks requiring high-intensity vigilance, the process from the subject receiving a stimulus, to the relevant brain regions (such as the brainstem) generating corresponding paradigmatic processing of the stimulus, and then to the related responses of other modal physiological signals (such as sudden heart rate jumps, muscle contractions for manipulation, and skin conductance stimulation under pressure), etc., can be captured and learned using multiple dynamic graph convolutional networks. This establishes a standard paradigm of multimodal physiological signals for subjects performing attention-related tasks, providing a relatively objective physical mechanism paradigm for patients with attention deficits during attention training and rehabilitation.

[0067] For multimodal signals of the same task, we first perform feature alignment on the multimodal signals through feature mapping, given the multimodal signal input. Where N mm Let be the total number of multimodal signal lead nodes, and r be the aligned feature dimension of each lead signal. Then the multimodal functional connection dynamic graph of this input is A. mm Where for p,q∈{1,…n mm},A mm-pq It can be calculated using equation (2):

[0068]

[0069] Where ω T ∈R F*1 For a learnable vector, ReLU(·) is the ReLU operation.

[0070] By combining the aforementioned dynamic maps of brain functional areas and dynamic maps of multimodal signal functional connectivity, a multi-region dynamic map convolutional network for attentional tasks is constructed.

[0071] (3) Feature extraction module

[0072] After constructing a multi-domain dynamic graph convolutional neural network for attention tasks, this example uses dynamic graph convolution to capture response paradigms and extract relevant features, thereby constructing more convincing brain region correlation paradigms and multimodal signal-behavioral response paradigms, laying the foundation for the next step of feature learning. Assume a graph A is given... k ∈R R*R ,in This represents the connection between lead i and lead j in the graph. We use a graph network layer to learn the connections between leads in this domain graph, A. k The regularization can be calculated using equation (3):

[0073]

[0074] in

[0075] Finally, the output of the l-th layer of the graph network learning layer in this domain graph is... It can be calculated using equation (4):

[0076]

[0077] in This is the output of the (l-1)th layer of the network for the k-th domain graph. W represents the feature output of the k-th domain graph after feature extraction. l graph ∈R F*h h is the mapping matrix of the l-th layer graph network of the k-th domain graph. l b is the feature output dimension of the k-th domain graph and the l-th layer graph network. l graph-k ∈R h*1 It is the bias vector of the l-th layer graph network of the k-th domain graph, and BN(·) is the batch regularization operation.

[0078] Finally, the outputs of the m-th (last) layer of the graph network for all domain graphs are concatenated to obtain the multi-domain graph representation OUTPUT obtained through the dynamic graph convolutional network. gra The calculation formula can be obtained from equation (5):

[0079]

[0080] CONCAT(·) is the splicing operation, FLATTEN(·) is the flattening operation, and K represents the total number of different domain diagrams.

[0081] (4) Feature learning module

[0082] This example uses an encoder framework based on a multi-head attention mechanism within the Transformer architecture to learn and emphasize relevance features extracted from dynamic graph convolutional networks, highlighting characteristics relevant to the target task. Assume there are n self-attention modules in total, and the input V of the i-th self-attention module... i V is the input to the (i-1)th self-attention module. i =Attention i-1 The output of the i-th self-attention module is Attention. i It can be calculated from equation (6):

[0083]

[0084] Among them, Q i ,K i V i Let be the feature vector of the query, key, and value in the i-th self-attention module, where sqrt(·) is the square root operation, and d o Indicates the dimension of the input features.

[0085] Finally, we obtain the Attention output from the Encoder, which consists of n self-attention modules. n This output serves as the basis for subsequent classification layers. The multi-head attention mechanism emphasizes and learns relevant features at different levels and dimensions through multiple self-attention structures, thereby highlighting features strongly related to the target task and reducing the influence of irrelevant features on the results. The feature learning module based on the multi-head attention mechanism can provide more objective feature indicators for the final attention paradigm discrimination.

[0086] (5) Note the assessment module

[0087] In the attention evaluation module, we use two fully connected neural networks to further integrate and reduce the features, finally providing a comprehensive evaluation result for different attention paradigms of the subjects. Let the weight matrix W1 and bias vector b1 of the first connected layer be W2 and the bias vector b2 of the second fully connected layer. Then the final model output OUTPUT... classify It can be expressed by equation (7):

[0088] OUTPUT classify =SOF TM AX(W2(ReLU(W1Attention n +b1))+b2) (7)

[0089] Ultimately, we used OUTPUT classify The output is used as a standard to evaluate the subject's current attentional state.

[0090] We use cross-entropy as the loss function of the model, and the model loss function can be expressed by equation (8):

[0091]

[0092] Where y i This represents the true output of the i-th sample. The model prediction for the i-th sample.

[0093] If the functions of this invention are implemented as software functional units and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] In summary, this invention provides a method and system for assessing attentional states based on brain-computer interfaces and multimodal signals. The attentional state assessment method includes a mapping stage, a feature extraction stage, a feature learning stage, and an attention assessment stage. The mapping stage is characterized by constructing dynamic brain functional area maps based on the dominant contributions of different brain regions to different tasks, and dynamic multimodal signal functional connectivity maps triggered by different brain functional areas and responding with physiological signals; and constructing multi-region dynamic map convolutional networks for attentional state tasks. This invention extracts multimodal signal collaboration paradigms with strong correlation characteristics related to the target attentional task as a classification basis, providing the contribution factors of EEG signals from different functional areas to the target attentional task, as well as the influence of correlation relationships between multimodal signals on the target attentional task, thereby improving the reliability and stability of brain-computer interface attention assessment.

[0095] Those skilled in the art should understand that variations can be implemented by combining existing technology with the above embodiments, which will not be elaborated here. Such variations do not affect the essence of the present invention, and will not be elaborated here either.

[0096] The preferred embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and the devices and structures not described in detail should be understood as being implemented in a conventional manner in the art. Any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention using the methods and techniques disclosed above, or modify them into equivalent embodiments with equivalent changes, without departing from the scope of the present invention. This does not affect the essential content of the present invention. Therefore, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the present invention's technical solutions still fall within the protection scope of the present invention.

[0097] Those skilled in the art will recognize that the units, i.e., algorithm steps, of the various examples described in connection with this embodiment can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. A method for assessing attentional state based on brain-computer interface and multimodal signals, comprising signal acquisition, feature extraction, feature learning, and attentional state assessment, characterized in that, It also includes a composition stage; the composition stage includes the following steps: Step S101: Design different attention evaluation subtask paradigms; Step S102: Construct a dynamic map of brain functional regions based on the dominant contributions of different brain regions to different tasks, and establish a feature paradigm for the target task based on EEG signals; construct a dynamic map of multimodal signal functional connectivity with different brain functional regions as triggers and other physiological signals as responses, to obtain the collaborative paradigm established by each EEG functional region and the response physiological signals for the target attention sub-task, the relationship between multimodal signals and attention tasks regarding action intentions and response behaviors, and the response mechanism from intention to behavior; combine the dynamic map of brain functional regions and the dynamic map of multimodal signal functional connectivity to construct a multi-region dynamic map convolutional network for attention state tasks; The feature extraction stage uses the multi-region dynamic graph convolutional network to extract relevant features from different experimental data of different target attention subtasks. The feature learning stage trains the model to learn the standard paradigm of different attentional state subtasks by training multiple subjects with features of subtasks under different attentional states. The attention assessment phase collects multimodal signals from individuals undergoing attention-related task training or rehabilitation via brain-computer interface. The standard paradigms for attention-related tasks learned through the brain functional area dynamic map and the multimodal signal functional connectivity dynamic map provide assessment criteria and effective guidance for patients. The dynamic brain function maps are defined using the Pearson correlation coefficient; given input ,in Let r represent the number of lead nodes in a single modality, and r represent the feature dimension of each lead signal. Then, the dynamic brain functional area map for this input is: Among them, for , Calculated by the following formula: ； In the dynamic diagram of the multimodal signal functional connection, for multimodal signals of the same task, the features of the multimodal signals are first aligned through feature mapping, and then the multimodal signal input is given. ,in Let be the total number of multimodal signal lead nodes, and r be the aligned feature dimension of each lead signal. Then the multimodal functional connection dynamic graph of this input is: Among them, for , Calculated by the following formula: ， in, For learnable vectors, This is a ReLU operation.

2. The method for assessing attentional state based on brain-computer interface and multimodal signals according to claim 1, characterized in that, The attention assessment subtask paradigm in step S101 assesses the subject's attentional ability based on five dimensions: attention selection, attention retention, attention allocation, attention control, and attention span.

3. The method for assessing attentional state based on brain-computer interface and multimodal signals according to claim 1, characterized in that, In step S102, the dynamic brain functional area map forms the main functional area by combining all the main leads in the prefrontal cortex of the brain. Other auxiliary functional areas are connected to the main functional area based on their relative positions and the degree of their auxiliary contribution to different target sub-tasks, that is, the graph adjacency relation matrix is ​​initialized.

4. The method for assessing attentional state based on brain-computer interface and multimodal signals according to claim 1 or 3, characterized in that, In step S102, the other physiological signals include electrocardiogram (ECG), electrodermal signal, and electromyography (EMG) signals.

5. An attention state assessment system based on brain-computer interface and multimodal signals, characterized in that, Implementing the attention state assessment method based on brain-computer interface and multimodal signals as described in any one of claims 1 to 4, comprising: The data acquisition module collects multimodal signals from subjects when performing attention-related tasks and constructs different attention assessment subtask paradigms. The mapping module constructs dynamic maps of brain functional areas based on the dominant contributions of different brain regions to different tasks, and constructs dynamic maps of multimodal signal functional connectivity triggered by different brain functional areas and responded to by physiological signals. The feature extraction module combines the dynamic maps of brain functional areas and the dynamic maps of multimodal signal functional connectivity to construct a multi-region dynamic map convolutional network for attentional state tasks, capture response paradigms, and extract relevant features. The feature learning module learns and trains the model to learn the standard paradigms of different attentional state subtasks by learning and training the multi-domain graph representations of multiple subjects in different attentional state subtasks. The attention assessment module further integrates and reduces the dimensionality of features through several fully connected neural networks, and finally provides a comprehensive assessment result of different attention paradigms of the subjects.

6. The attention state assessment system based on brain-computer interface and multimodal signals according to claim 5, characterized in that, The feature learning module employs an Encoder framework based on a multi-head attention mechanism within the Transformer architecture to learn and emphasize the relevance characteristics extracted by the multi-region dynamic graph convolutional network, resulting in an Encoder output composed of n self-attention modules. ; Input of the i-th self-attention module The input to the (i-1)th self-attention module is... , No. The output of each self-attention module It can be calculated using the following formula: ， in, , , For the first The query, key, and value feature vectors in each self-attention module, where For the square root operation, Indicates the dimension of the input features.

Images