Cognitive disorder evaluation system and method based on emotional interaction multi-mode physiological signals

Abstract

The invention provides a cognitive impairment assessment system and method based on emotion interaction multi-mode physiological signals. The cognitive impairment assessment system comprises a multi-mode physiological signal acquisition module, an emotion interaction module based on emotion recognition and a cognitive impairment assessment module. The multi-modal physiological signal acquisition module acquires multi-dimensional physiological signals such as electroencephalogram, electro-oculogram and electrocardio through a flexible sensor, and a physiological signal coupling network is constructed; the emotion interaction module based on emotion recognition adopts a heterogeneous graph neural network to analyze emotion response under visual stimulation, recognizes deviation between emotion and a preset stimulation type through a comparison model, dynamically optimizes visual stimulation parameters, and forms closed-loop emotion interaction; and the cognitive impairment evaluation module combines an SE-CNN deep learning model and fuses multi-modal physiological data and emotion interaction characteristics to realize accurate classification of cognitive impairment. According to the method, the accuracy and individualized efficiency of emotion recognition can be effectively improved, and a new normal form and a new method are provided for early cognitive impairment assessment.

Description

Technical Field

[0001] The present invention relates to a cognitive impairment assessment system and method based on emotion interaction multi-modal physiological signals, and belongs to the technical fields of test and measurement technology and instrument technology. Background Art

[0002] Neurodegenerative diseases such as Alzheimer's disease, Parkinson's disease, and multiple sclerosis impose a heavy burden on individuals, families, and society. Cognitive impairment / dysfunction is one of the main clinical manifestations of the above neurodegenerative diseases, manifested as a decline in abilities such as memory, attention, perception, understanding and judgment, and executive ability. At present, the pathogenesis of brain diseases related to cognitive impairment / dysfunction is not yet clear, and there are lack of effective early diagnosis and intervention means. Therefore, realizing early screening and early intervention for cognitive impairment / dysfunction is an effective means to improve the condition of patients with neurodegenerative diseases, and has important clinical significance and social effects.

[0003] Currently, the clinical mainly uses the cognitive task scoring method of scale questionnaires, such as the Montreal Cognitive Assessment (MoCA), the Mini-Mental State Examination (MMSE), etc. The total score of the assessment has high sensitivity for the screening of cognitive impairment, but the accuracy of the assessment of some independent cognitive functions such as attention, perception, and executive ability is not high. Therefore, the traditional scale assessment method has problems such as low intelligence level, heavy burden on patients, incomplete cognitive function assessment, and poor objectivity. More importantly, scale assessment is likely to increase the psychological pressure of patients, resulting in problems such as poor compliance and high dropout rate, reducing the efficiency of clinical diagnosis and early screening of diseases.

[0004] Previous scientific research and clinical practice have shown that emotional changes are closely related to people's normal cognitive activities, and emotion-oriented assessment and intervention methods can have a positive effect on disease diagnosis and treatment. Therefore, monitoring multi-dimensional information such as physiological signals in the cognitive processing process of patients through emotion interaction technology, accurately analyzing and feedback-regulating the changes in their emotional states provide a new and effective way for the early screening, accurate diagnosis, and intervention treatment of cognitive impairment / dysfunction.

[0005] Currently, there are few reports on the diagnosis and treatment technology of cognitive impairment / dysfunction based on emotion interaction. Technologies such as multi-modal emotion databases and emotion computing models around neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease are still in the primary research stage. Therefore, it is proposed to study a cognitive impairment assessment technology based on emotion interaction multi-modal physiological signals, focusing on neurodegenerative diseases such as Alzheimer's disease and Parkinson's disease. Based on multi-modal physiological signals, considering multiple aspects such as high comfort, high accuracy, high synchronization, and multi-scene adaptability, a multi-scene multi-modal emotion database is constructed, multi-perspective heterogeneous information fusion emotion recognition technology is studied, and a cognitive function assessment model based on multi-modal emotion interaction is established. Summary of the Invention

[0006] Technical Problem:

[0007] The present invention needs to solve the problems existing in the existing cognitive dysfunction evaluation methods, such as low intelligence level, heavy burden on patients, incomplete evaluation, poor objectivity, and lack of effective early diagnosis and intervention means. It is expected to achieve the goal of early screening, accurate diagnosis and intervention treatment of cognitive function impairment / dysfunction by constructing an early cognitive dysfunction evaluation system based on emotion interaction multi-modal physiological signals.

[0008] Technical Solution:

[0009] The present invention provides a cognitive disorder evaluation system and method based on emotion interaction multi-modal physiological signals. The system includes a wearable multi-modal physiological signal acquisition module, a multi-modal heterogeneous information fusion emotion recognition module, and a cognitive function evaluation module based on emotion interaction. The acquisition module integrates multi-modal sensors such as electroencephalogram and electrocardiogram, preprocesses the signals and extracts features such as spectral power, amplitude variance, and inter-organ coupling strength; the emotion recognition module induces emotional responses through visual stimuli, processes single-modal signals using the SE-CNN module, and constructs a heterogeneous graph neural network to fuse multi-modal heterogeneous information; the cognitive function evaluation module uses networks such as GRU to perform temporal modeling on the fused features, outputs the cognitive disorder level (mild, moderate, severe), and dynamically adjusts the stimulation parameters according to the evaluation results to form a "stimulation-response-feedback" closed loop, and analyzes the cross-organ association in combination with the physiological signal coupling network to achieve accurate evaluation of cognitive disorders based on emotion interaction.

[0010] The present invention adopts the following technical solutions:

[0011] A cognitive disorder evaluation system based on emotion interaction multi-modal physiological signals, comprising: a wearable multi-modal physiological signal acquisition module, a multi-modal heterogeneous information fusion emotion recognition module, and a cognitive function evaluation module based on emotion interaction. The wearable multi-modal physiological signal acquisition module includes a wearable multi-modal physiological signal acquisition unit, an inter-organ coupling analysis unit based on time-delay stability, and a physiological signal coupling network unit; the multi-modal heterogeneous information fusion emotion recognition module includes a heterogeneous graph neural network unit, an emotion recognition unit, and an emotion visual stimulation unit; the cognitive function evaluation module based on emotion interaction includes a multi-modal physiological signal unit, an SE-CNN network model unit, and a cognitive disorder level unit.

[0012] As a further improvement of the present invention, the physiological signals include electroencephalogram, electrocardiogram, electromyogram, electrooculogram, respiration, pulse, etc., and fuse multiple modalities.

[0013] As a further improvement of the present invention, the wearable multi-modal physiological signal acquisition module collects multi-modal physiological signals such as electroencephalogram (EEG), electrocardiogram (ECG), electrooculogram (EOG), respiration (RESP), electromyogram (EMG), and photoplethysmogram (PPG) through wearable devices. Signal acquisition is carried out under emotional visual stimulation, and special sequence extraction and normalization processing are performed on the collected multi-modal physiological signals. By judging the time lag stability, the coupling strength between organs is obtained, and the quantitative analysis of the coupling relationship between different physiological systems is realized, and a physiological signal coupling network is constructed. Among them, the condition for judging the time lag stability is: if the standard deviation of the maximum position of the cross-correlation function in at least 4 consecutive time windows out of 5 consecutive time windows is ≤ ±1, it is considered stable.

[0014] Specifically, the 60s long-term signal is divided into 2s sliding windows (step size 1s) through data segmentation to generate overlapping data segments; the features of each collected physiological signal are extracted, and the spectral power of each frequency band (δ, θ, α, σ, β) of the electroencephalogram, the variance of the electrooculogram amplitude, the respiration rate sequence, the heart rate sequence, the variance of the electromyogram amplitude, the pulse rate sequence, etc. are calculated; Z-score normalization is performed on the features of each modality to eliminate the dimension difference; based on the cross-correlation function, the time lag stability (TDS) between organs is calculated, and the coupling strength %TDS = stable TDS duration / total duration is defined to construct a multi-modal physiological signal coupling network.

[0015] As a further improvement of the present invention, the modality heterogeneous information fusion emotion recognition module uses emotional visual stimulation to adjust and change the emotional state of the subject; through video or other visual stimulation methods, different emotions of the subject such as happiness, anger, sadness, disgust, fear, and surprise are stimulated, so as to collect multi-modal physiological signals in different emotional states; by analyzing the dynamic change characteristics of physiological signals in different emotional states, the recognition of the emotional state of the subject is realized, providing emotional state information for subsequent cognitive impairment assessment; the graph transformer network (GTN) is used to automatically extract meta-paths from heterogeneous graphs, that is, the relationships between different channels, so as to model the heterogeneity of multi-modal data. The values of all channels are used as node features and adjacency matrices to form a heterogeneous graph, capturing the relevance and interaction between different physiological signal channels, and providing a feature representation in the spatial domain for subsequent emotion recognition. After the multi-modal heterogeneous information fusion emotion recognition module constructs the physiological signal coupling network, it extracts meta-paths through GTN to capture the heterogeneity of multi-modal physiological signals; then uses the graph convolutional network GCN to extract spatial domain features, processes graph structure data, and captures the spatial relationship between nodes and their neighbor nodes; then models the time domain features through GRU, processes sequence data and captures the temporal dependence relationship, and finally fuses the spatial and temporal features for emotion classification recognition.

[0016] Specifically, the heterogeneous graph neural network is input into the graph transformation network (GTN), and path features under different semantics are extracted according to predefined meta-paths; the extracted meta-path features are fed into the graph convolutional network (GCN) to aggregate the features of the node itself and its neighbor nodes, so as to extract the features of each node in the spatial domain, providing information in the spatial dimension for subsequent sentiment analysis; the gated recurrent unit (GRU) is a neural network structure for processing sequence data, which can model temporal information and extract the temporal features of nodes; then the extracted spatial domain features and temporal features are fused to combine the two different-dimensional feature information to form a more comprehensive node representation; then the fused features are input into a classifier, and the classifier classifies the nodes according to these features, and finally outputs the classification result.

[0017] As a further improvement of the present invention, the cognitive function evaluation module based on emotional interaction inputs the multi-modal physiological signal data of the individual to be evaluated into the trained SE-CNN network model, and the model finally outputs the evaluation result of the cognitive impairment level of the individual through feature extraction, fusion and classification operations.

[0018] Specifically, various physiological signals are respectively input into multiple SE-CNN modules, and each physiological signal corresponds to an SE-CNN module for extracting its features. Among them, each SE-CNN module includes multiple 1D-CNN layers for capturing features in different dimensions of physiological signals; the BN-PReLU layer is used for normalization and non-linear activation to enhance the expression ability and stability of the model; the Squeeze-and-Excitation (SE) module dynamically adjusts the weights of feature channels to highlight important features and suppress unimportant features; the Attention mechanism focuses on the key parts in the signal to improve the attention to important features; the Dropout layer (Dropout rate is 0.5) is used to prevent overfitting and improve the generalization ability of the model. Then, the feature vectors extracted by each SE-CNN module are input into the fully connected layer to fuse the features from different physiological signals, and the fused feature vectors are classified through the softmax layer and mapped to the corresponding cognitive impairment level categories (levels I, II, and III). Finally, the probability distribution of each cognitive impairment level is output as the evaluation result.

[0019] Beneficial effects:

[0020] 1. High - comfort multi - modal data acquisition technology: Adopting flexible anhydrous dry electrodes and lightweight design, it supports continuous wearing for a long time (≥8 hours), solves the problems of traditional gel electrode allergies, signal drift, and wearing discomfort, and significantly improves user compliance; combined with an adaptive impedance adjustment circuit, it automatically compensates for signal attenuation when the electrode - skin slides, improves the signal - to - noise ratio of physiological signals such as electroencephalogram (EEG) and electrocardiogram (ECG), and provides high - quality data support for subsequent emotion recognition and cognitive assessment.

[0021] 2. Multi - modal heterogeneous information fusion ability: Based on time - delay stability (TDS), a dynamic physiological interaction network is constructed to quantify for the first time the coupling relationships among systems such as the brain, heart, and respiration, and effectively capture the dynamic correlations between organs during emotional state transitions (such as the enhanced coupling between EEG delta waves and low - frequency components of heart rate variability in a sad state); through heterogeneous graph neural networks (GTN / GCN / GRU), the heterogeneity and temporal features of multi - modal signals are fused. In the scenario of single - modality loss (such as only collecting ECG / EEG signals), the emotion recognition accuracy still remains ≥85%, significantly reducing the model's dependence on a single modality and improving the robustness in complex clinical scenarios.

[0022] 3. Precise cognitive function evaluation system: A fusion evaluation model of "traditional scale + multi - modal physiological signals" is constructed. For dimensions such as memory, attention, and executive function, combined with physiological indicators such as EEG power spectrum and ECG RR interval and MoCA / MMSE scale scores, a multi - dimensional evaluation system is formed; the SE - CNN attention mechanism is introduced to dynamically suppress irrelevant features such as motion artifacts and noise, highlighting key indicators such as autonomic nerve function and EEG rhythm, and realizing precise early screening and grading diagnosis.

[0023] 4. Dynamic emotional interaction closed - loop intervention: Based on real - time emotion recognition results, video stimuli (encouraging / comforting content) are automatically triggered to regulate the patient's emotions, forming a closed - loop of "emotion mobilization - signal acquisition - cognitive assessment", effectively relieving the psychological pressure brought by traditional scale evaluations and reducing the patient dropout rate; through a visual interface, the emotional state and cognitive assessment report are displayed in real - time, supporting early warning of abnormal states, and providing data - driven decision - making support for personalized intervention treatment.

[0024] 5. Innovative breakthrough in technical paradigm: Breaking through the limitation of traditional cognitive assessment relying on subjective scales, establishing an association model of "emotional interaction - physiological signal - cognitive function", providing a new paradigm for the early diagnosis of neurodegenerative diseases (such as Alzheimer's disease and Parkinson's disease); through the construction of a multi - modal physiological network and deep - learning algorithms, the potential mechanisms of cognitive and emotional interactions are explored, providing theoretical support for the study of pathogenesis and the optimization of intervention strategies, and having important clinical application value and social significance. Description of the Drawings

[0025] Figure 1It is a schematic diagram of the technical route structure of the present invention.

[0026] Figure 2 It is a schematic diagram of the process for collecting multi-modal physiological signals in the present invention.

[0027] Figure 3 It is a schematic diagram of the process for emotion classification in the present invention.

[0028] Figure 4 It is a design diagram of a cognitive impairment assessment model based on SE-CNN in the present invention. Detailed implementation manners

[0029] The present invention will be further clarified below in conjunction with the accompanying drawings and detailed implementation manners. It should be understood that the following detailed implementation manners are only used to illustrate the present invention and not to limit the scope of the present invention.

[0030] Example 1

[0031] As Figure 1 shown, the present invention provides a cognitive impairment assessment system based on emotion interaction multi-modal physiological signals, including: a wearable multi-modal physiological signal acquisition module 1, a multi-modal heterogeneous information fusion emotion recognition module 2, and a cognitive function assessment module 3 based on emotion interaction. The physiological signal coupling network formed by the wearable multi-modal physiological signal acquisition module 1 serves as the input of the multi-modal heterogeneous information fusion emotion recognition module 2; at the same time, the emotional visual stimulus provided by the multi-modal heterogeneous information fusion emotion recognition module 2 is input into the wearable multi-modal physiological signal acquisition module 1; the data and emotion recognition results processed by the multi-modal heterogeneous information fusion emotion recognition module 2 serve as the input of the cognitive function assessment module 3 based on emotion interaction; the cognitive function assessment module 3 based on emotion interaction combines traditional cognitive test scales and physiological signal data during the emotion interaction process, and finally outputs a grade assessment of cognitive impairment.

[0032] The wearable multi-modal physiological signal acquisition module 1 includes a wearable multi-modal physiological signal acquisition unit 11, an inter-organ coupling analysis unit 12 based on time-delay stability, and a physiological signal coupling network unit 13.

[0033] The multi-modal heterogeneous information fusion emotion recognition module 2 includes an emotional visual stimulus unit 21, an emotion recognition unit 22, and a heterogeneous graph neural network unit 23; the wearable multi-modal physiological signal acquisition unit 11, the inter-organ coupling analysis unit 12 based on time-delay stability, the physiological signal coupling network unit 13, the heterogeneous graph neural network unit 23, the emotion recognition unit 22, and the emotional visual stimulus unit 21 are connected in sequence, the emotional visual stimulus unit 21 is connected to the wearable multi-modal physiological signal acquisition unit 11, and the emotion recognition unit 22 is connected to the cognitive function assessment module 3 based on emotion interaction.

[0034] The cognitive function evaluation module 3 based on emotional interaction includes a multimodal physiological signal unit 31, an SE-CNN network model unit 32, and a cognitive impairment level unit 33; the multimodal physiological signal unit 31, the SE-CNN network model unit 32, and the cognitive impairment level unit 33 are connected in sequence.

[0035] As Figure 2 shown, the wearable multimodal physiological signal acquisition module 1 constructs a physiological signal complex network through the acquisition, preprocessing, feature extraction, and coupling strength analysis of multimodal physiological signals.

[0036] Specifically, it includes the steps:

[0037] Step 111, Multimodal physiological signal acquisition: Simultaneously acquire multiple physiological signals through wearable devices, including electroencephalogram (EEG), electrooculogram (EOG), respiration (RESP), electrocardiogram (ECG), electromyogram (EMG), and photoplethysmogram (PPG) signals.

[0038] Step 112, Feature sequence extraction and normalization: Segment the collected continuous signals into segments with a fixed duration of 60 s, extract multiple feature sequences from the segmented signals, such as the spectral power of EEG (including δ, θ, α, β frequency bands), the amplitude variance of EOG, the respiration rate sequence of RESP, the heart rate sequence of ECG, the amplitude variance of EMG, the pulse rate sequence of PPG, etc., and then perform normalization processing on the extracted feature sequences to make them have zero mean and unit standard deviation, eliminate the dimensional differences between different features, and improve the accuracy of subsequent analysis.

[0039] Step 12, Inter-organ coupling strength analysis: Calculate the cross-correlation function Cx,y(τ) between different physiological signals to measure the interaction strength between different organs; extract the time lag parameter τ0 from the cross-correlation function, which represents the time delay relationship between different signals and reflects the communication delay between organs; determine the stability of the time lag through a series of calculation and judgment criteria, screen out the stable TDS part for subsequent analysis; calculate the proportion of stable TDS in the total TDS (%TDS), which reflects the tightness of coupling between different organs.

[0040] Step 13, Physiological signal complex network construction: Construct different physiological signals and their relationships into a complex network, where the nodes in the network represent different physiological signals (such as eyes, brain, lungs, heart, muscles, blood vessels, etc.), the edges represent their coupling relationships, and the strength of the edges is determined by indicators such as %TDS.

[0041] As Figure 3As shown, in the emotion classification step, by combining various neural network structures such as GTN, GCN, and GRU, making full use of the diversity of nodes and relationships in graph-structured data, as well as time series information, etc., the effective extraction and classification of emotional information from multimodal physiological signals are realized.

[0042] Specifically, Figure 3 It includes the steps:

[0043] Step 13: Construction of the physiological signal complex network.

[0044] Step 231: Meta-path extraction. The heterogeneous graph is input into the graph transformation network (GTN), and GTN extracts path features under different semantics according to the predefined meta-paths.

[0045] Step 232: Spatial domain feature extraction. The extracted meta-path features are fed into the graph convolutional network (GCN). GCN performs convolutional operations on the graph structure, aggregating the features of the node itself and its neighbor nodes, thereby extracting the features of each node in the spatial domain, reflecting the position of the node in the graph structure and the comprehensive influence of its surrounding neighbor nodes.

[0046] Step 233: Temporal domain feature extraction. The gated recurrent unit (GRU) models the temporal information and extracts the feature change law of the nodes in the time dimension.

[0047] Step 221: Feature fusion and classification. The extracted spatial domain features and temporal domain features are fused to form a more comprehensive node representation, integrating the information of the nodes in the spatial and time dimensions. The fused features are input into the classifier, and the classifier classifies the nodes according to these features.

[0048] Step 222: Finally, output the classification results of different emotion categories (such as positive, negative, neutral, etc.).

[0049] Such as Figure 4 As shown, the cognitive function evaluation module 3 based on emotional interaction makes full use of various neural network structures of the SE-CNN module to fully mine the emotional interaction features in each physiological signal, providing a comprehensive and accurate basis for cognitive function evaluation.

[0050] Specifically, it includes the steps:

[0051] Step 31: Multiple physiological signals (such as electroencephalogram, electrocardiogram, electromyogram, etc.) are used as inputs, and each signal enters an SE-CNN module.

[0052] Step 321: Each physiological signal is subjected to feature extraction through the SE-CNN module. The SE-CNN module consists of multiple 1D-CNN layers, BN-PReLU layers, Squeeze-and-Excitation modules, Attention layers, and Dropout layers. The 1D-CNN layers extract local features of the signal; the BN-PReLU layers normalize and nonlinearly activate the features to enhance the expressive ability of the model; the Squeeze-and-Excitation modules dynamically adjust the weights of the feature channels to highlight important features; the Attention layers focus on the key parts in the signal to improve the attention to important features; the Dropout layers prevent overfitting and improve the generalization ability of the model.

[0053] Step 322: The feature vectors extracted by each SE-CNN module are input into the fully connected layer, and the fully connected layer fuses the features from different physiological signals.

[0054] Step 323: The fused feature vectors are classified through the softmax layer to convert the numerical values output by the model into a probability distribution.

[0055] Step 33: Map the fused feature vectors to the corresponding cognitive impairment level categories (such as levels I, II, and III).

[0056] The technical means disclosed in the solution of the present invention are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that for those of ordinary skill in the art in this technical field, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A cognitive impairment assessment system based on emotional interaction multimodal physiological signals, characterized in that: It includes a wearable multimodal physiological signal acquisition module, a multimodal heterogeneous information fusion emotion recognition module, and a cognitive function assessment module based on emotion interaction, wherein the wearable multimodal physiological signal acquisition module is used to form a physiological signal coupling network and serve as the input of the multimodal heterogeneous information fusion emotion recognition module; the emotional visual stimulation provided by the multimodal heterogeneous information fusion emotion recognition module is input into the wearable multimodal physiological signal acquisition module; the data processed by the multimodal heterogeneous information fusion emotion recognition module and the emotion recognition results are used as the input of the cognitive function assessment module based on emotion interaction; the cognitive function assessment module based on emotion interaction combines the traditional cognitive test scale and the physiological signal data in the process of emotion interaction, and finally outputs the level assessment of cognitive impairment.

2. The cognitive impairment assessment system based on emotional interaction multimodal physiological signals according to claim 1 is characterized in that: The wearable multimodal physiological signal acquisition module collects EEG, ECG, EOG, RESP, EMG, and PPG multimodal physiological signals through wearable devices, performs signal acquisition under emotional visual stimulation, performs special sequence extraction and normalization processing on the collected multimodal physiological signals, and obtains the coupling strength between organs by judging the time delay stability, realizes quantitative analysis of the coupling relationship between different physiological systems, and constructs a physiological signal coupling network, wherein the condition for judging the time delay stability is: the standard deviation of the maximum position of the cross-correlation function of at least 4 consecutive time windows in 5 consecutive time windows is ≤±1, which is considered to be stable.

3. The cognitive impairment assessment system based on emotional interaction multimodal physiological signals according to claim 1 is characterized in that: The multimodal heterogeneous information fusion emotion recognition module uses emotional visual stimulation to regulate and change the emotional state of the subject; Through videos or other visual stimulation methods, the subjects' different emotions such as happiness, anger, sadness, disgust, fear, and surprise are stimulated, so as to collect multimodal physiological signals under different emotional states; by analyzing the dynamic change characteristics of physiological signals under different emotional states, the emotional state of the subjects can be identified, and emotional state information can be provided for subsequent cognitive impairment assessment; the graph transformer network GTN is used to automatically extract meta-paths from the heterogeneous graph, that is, the relationship between different channels, to model the heterogeneity of multimodal data, and the values ​​of all channels are used as node features and adjacency matrices to form a heterogeneous graph, which captures the correlation and interaction between different physiological signal channels and provides feature representation in the spatial domain for subsequent emotion recognition.

4. The cognitive impairment assessment system and method based on emotional interaction multimodal physiological signals according to claim 3 is characterized in that: After the multimodal heterogeneous information fusion emotion recognition module completes the construction of the physiological signal coupling network, the meta-path is extracted through GTN to capture the heterogeneity of multimodal physiological signals; the graph convolutional network GCN is then used to extract spatial domain features, process graph structure data, and capture the spatial relationship between nodes and their neighbor nodes; then the time domain features are modeled through GRU, sequence data is processed and temporal dependencies are captured, and finally the spatial and temporal features are fused for emotion classification and recognition.

5. The cognitive impairment assessment system based on emotional interaction multimodal physiological signals according to claim 1, characterized in that: The emotional interaction-based cognitive function assessment module inputs the multimodal physiological signal data of the individual to be assessed into a trained SE-CNN network model, and the model finally outputs the cognitive impairment level assessment result of the individual through feature extraction, fusion and classification operations.

6. The cognitive impairment assessment system based on emotional interaction multimodal physiological signals according to claim 5 is characterized in that: The cognitive function assessment module based on emotional interaction has an independent SE-CNN module for feature extraction and processing for each physiological signal. The SE-CNN module includes a 1D-CNN layer for extracting temporal features in physiological signals, a BN-PreLU layer for accelerating training and introducing nonlinearity, and an SE unit for recalibrating feature weights. The Attention mechanism is used to further enhance the expression ability of important features. In order to prevent overfitting, the Dropout layer is used to randomly discard features. The data is then passed to the fully connected layer to integrate features from different physiological signals to form a comprehensive feature representation. Finally, the fused features are classified through the Softmax layer to output the level of cognitive impairment.

7. A method for assessing cognitive impairment based on multimodal physiological signals of emotional interaction, characterized in that: A system based on any one of claims 1-6, comprising: a multimodal physiological signal acquisition module that acquires multi-channel synchronous physiological signals such as human brain waves, eyeballs, electrocardiograms, respiration, pulses and myoelectricity, and constructs a physiological signal coupling network that reflects the interaction of physiological systems; an emotion interaction module based on emotion recognition that uses a heterogeneous graph neural network to identify the emotional state represented by the physiological signal coupling network under emotional visual stimulation, and compares the types of emotions and visual stimulations recognized by the model to dynamically update the type and intensity of visual stimulation to achieve human-computer emotional interaction in the cognitive assessment process; a cognitive dysfunction assessment module that combines the patient's emotional interaction characteristics and multimodal physiological signal data in the cognitive assessment and treatment process, proposes a cognitive assessment method based on SE-CNN, and achieves accurate assessment of cognitive dysfunction.

Images