Cognitive disorder closed-loop evaluation system and method based on multi-physiological parameter coupling in multi-source collaborative stimulation

Abstract

The invention discloses a cognitive impairment closed-loop evaluation system and method based on multi-physiological parameter coupling in multi-source collaborative stimulation. The cognitive impairment closed-loop evaluation system comprises a multi-modal physiological signal acquisition module, a stimulation efficiency evaluation module and a closed-loop cognitive impairment evaluation module. The multi-modal physiological signal acquisition module acquires multi-source signals such as electroencephalogram, electrocardio and myoelectricity in rehabilitation training of a patient in real time, and constructs a brain-peripheral organ physiological coupling network; the stimulation efficiency evaluation module extracts the time-frequency-space characteristics of the physiological signals by using a space-time attention network so as to evaluate the efficiency of nerve regulation / sensory stimulation, thereby dynamically optimizing nerve regulation parameters and training difficulty; and the closed-loop cognitive impairment evaluation module is used for modeling dynamic association of a physiological network sequence through a heterograph recursive network, and realizing cognitive impairment grading in an acquisition-evaluation-regulation closed loop. The problems that in the cognitive rehabilitation training process of old people, equipment comfort is low, wearing is complex, physiological signal fusion is difficult, and stimulation efficiency evaluation is inaccurate are solved.

Description

Technical Field

[0001] The present invention relates to a closed-loop evaluation system and method for cognitive impairment based on the coupling of multiple physiological parameters in multi-source collaborative stimulation, belonging to the fields of signal detection and analysis and medical rehabilitation equipment. Background Art

[0002] Geriatric cognitive impairment is a neurodegenerative disease characterized by memory decline and reduced executive function, including different stages such as subjective cognitive decline (SCD), mild cognitive impairment (MCI), and Alzheimer's disease (AD). Such diseases seriously affect the quality of life of patients and result in high social and medical burdens. In existing rehabilitation technologies, although the evaluation and training system based on artificial intelligence can partially improve the rehabilitation efficiency, there are still significant technical defects: firstly, the sensory stimulation mode is single, and the multi-channel collaborative intervention of vision, hearing, touch, etc. is insufficient, resulting in limited activation effect of neural plasticity; secondly, the physiological parameter detection module and the rehabilitation stimulation module are designed separately, with a high system response delay and unable to achieve real-time matching of the training task difficulty and the patient's cognitive state; thirdly, traditional devices are bulky, have poor wearing comfort, and have a complex interaction interface, resulting in low compliance of elderly patients. The current cognitive function assessment methods mainly rely on scale scoring or single physiological signal (such as electroencephalogram) analysis, and fail to effectively integrate the coupling mechanisms of multiple organs such as the brain, heart, and muscle and behavioral data, resulting in one-sided assessment results. For example, electroencephalogram signals are easily interfered by movement artifacts, and single-modal analysis is difficult to distinguish AD from vascular cognitive impairment; the dynamic correlation mechanisms between peripheral physiological parameters such as electrocardiogram and electromyogram and the central cognitive state have not been fully explored. In addition, existing systems lack a closed-loop feedback mechanism, the training plan is fixed, and it is unable to dynamically adjust the stimulation intensity and task complexity according to the patient's real-time cognitive state, seriously affecting the individualization and accuracy of the rehabilitation process.

[0003] In recent years, wearable physiological monitoring devices have become a research hotspot due to their advantages such as lightweight and high comfort. However, their application in the field of cognitive rehabilitation still faces the following challenges: 1) Insufficient ability to synchronously collect and fuse multi-modal signals, and no brain-peripheral organ coupling network model has been established; 2) Existing algorithms mostly extract features based on single temporal or spatial dimensions, making it difficult to capture the spatio-temporal dynamic correlations of multiple physiological parameters; 3) The rehabilitation training and evaluation links are fragmented, and no closed-loop optimization system of "monitoring - analysis - intervention - re-evaluation" has been formed. Especially for patients in different stages of SCD, MCI, and AD, existing technologies have failed to design differentiated collaborative stimulation strategies, resulting in limited early intervention effects. Therefore, there is an urgent need to develop a cognitive rehabilitation system that integrates multi-physiological parameter detection, multi-source collaborative stimulation, and closed-loop adaptive strategies. Achieve non-invasive and long-term physiological monitoring through lightweight flexible sensing technology; construct a highly immersive training environment in combination with virtual reality; use organ coupling networks and spatio-temporal convolutional algorithms to quantify cognitive function changes; finally establish a closed-loop rehabilitation paradigm of "physiological feedback - stimulation regulation - model iteration". This invention is based on the national health cause, breaks through the key scientific problems of virtual reality cognitive disorder diagnosis and treatment, tackles the key technologies of multi-sensory collaborative stimulation intervention and cognitive diagnosis and treatment, and constructs an interactive diagnosis and treatment prototype for cognitive disorders. It will further promote the popularization and application of artificial intelligence and digital diagnosis and treatment equipment achievements in clinical practice, improve the research level of cognitive disorder assessment and training in China, and strengthen the independent innovation ability of Chinese medical device enterprises. Summary of the Invention

[0004] Technical Problem: The present invention aims to solve the problems of low device comfort, complex wearing, difficult physiological signal fusion, and inaccurate stimulation efficacy evaluation during the cognitive rehabilitation training of the elderly. In the prior art, cognitive rehabilitation training devices often have cumbersome wearing and redundant steps, resulting in low acceptance by users; there are modal differences and noise problems in multi-modal physiological signal fusion, making it difficult to effectively integrate; most of the stimulation efficacy evaluation methods only consider single features, ignoring the complementarity of temporal, spatial, and spectral features; and there is a lack of training strategies for real-time feedback and dynamic adjustment, unable to meet the personalized needs of the elderly in different cognitive states.

[0005] Technical Solution: The present invention solves the above problems through the following technical solutions:

[0006] A closed-loop evaluation system for cognitive disorders based on multi-physiological parameter coupling in multi-source collaborative stimulation, comprising:

[0007] (1) A multi-physiological parameter monitoring and multi-source collaborative stimulation rehabilitation system

[0008] A multi-modal physiological signal acquisition module with lightweight flexible design: Aiming at the high comfort requirements of the elderly for the device, study the wearable and portable design of acquisition devices for few-channel electroencephalogram, electrocardiogram, pulse, respiration, etc., optimize the wearing position, method, and structure, reduce the device weight, and improve comfort.

[0009] Integrated multi-source stimulation and evaluation system: Research questionnaire scales, task performance evaluation and aging-friendly interactive interfaces, and combine data analysis technology to build a data portrait of the cognitive function of the elderly; integrate multimodal physiological and behavioral acquisition technology modules with multi-source stimulation modules, design highly comfortable, non-interference and conflict-free wearable physiological and behavioral parameter acquisition modules, non-contact signal acquisition modules and lightweight multi-sensory collaborative stimulation virtual reality modules, and realize system integration.

[0010] (2) Stimulation efficacy evaluation based on organ coupling network and spatiotemporal convolution

[0011] Multimodal physiological signal data segmentation and feature extraction: The acquired multimodal physiological signals are segmented, and then the highly specific indicators of EEG, EKG, EMG, EG, eye movement and other signals are mined from the physiological level to achieve feature extraction and normalization of the data. A specific algorithm is used to determine the coupling parameters between organs, and then a complex physiological signal network is constructed. At the same time, the local and global feature patterns of various physiological signals are adaptively extracted by combining multi-scale convolution and self-attention feature fusion methods to achieve emotion recognition of the subject.

[0012] Cognitive paradigm design and multimodal fusion: Design a variety of cognitive paradigms (contextual memory, executive function, spatial cognition, attention function, verbal communication, comprehension and decision-making, motor balance), monitor the EEG data in the resting state, during stimulation and task state before and after stimulation in real time, measure the resting motor threshold, TMS-evoked potential and physiological and behavioral characteristics of the corresponding cognitive paradigm before and after stimulation, and improve the evaluation effect of changes in cognitive function under sensory stimulation through the multimodal fusion method of modal cross-attention mechanism.

[0013] (3) Cognitive function assessment based on closed-loop adaptive rehabilitation strategy

[0014] Search for specific digital biomarkers: Based on new discoveries about the physiological changes of cognitive function under multi-sensory stimulation, this study seeks specific digital biomarker combinations that are highly correlated with changes in cognitive function.

[0015] Personalized cognitive training program: In-depth analysis of the training effects of elderly people with different cognitive states under different stimulation paradigms and different training loads, and the introduction of new multi-sensory synergistic stimulation intervention technology; designing training strategies that match cognitive functions with multi-sensory synergistic stimulation training paradigms, real-time feedback on cognitive load and dynamic adjustment of task difficulty, and constructing a cognitive function evaluation model based on multiple physiological parameters.

[0016] Specifically, it includes: a multi-source collaborative stimulation module, a multi-modal physiological signal acquisition module, a stimulation efficacy evaluation module, and a closed-loop cognitive impairment evaluation module. Among them, the multi-source collaborative stimulation module: includes a multi-sensory stimulation unit driven by virtual reality and an age-friendly interaction interface, where the senses include vision, hearing, touch, vestibule, and proprioception, and is used to generate an immersive scene matching the cognitive training paradigm; the multi-modal physiological signal acquisition module: uses a lightweight flexible wearable device to realize the acquisition of electroencephalogram (EEG), electrocardiogram (ECG), electromyogram (EMG), galvanic skin response (GSR), and eye movement signals. Through ergonomic design, the wearing position and structure are optimized to reduce weight, realizing non-invasive and low-load real-time synchronous acquisition of physiological signals. Then, for the collected physiological data, a dynamic correlation network of physiological signals between the brain center and peripheral organs is constructed based on the time-delay stability (TDS) algorithm. Taking the activities of the brain, heart, respiration, eyes, and muscles as nodes and the time-delay correlation as edges, the network topology structure of the cooperative action of multiple organs is quantified; the stimulation efficacy evaluation module: uses a spatio-temporal attention dense network to synchronously extract the time-frequency-space multi-dimensional features of physiological signals, combines the modal cross-attention mechanism to fuse physiological data and behavioral task performance data, and generates a cognitive load quantification index; based on the obtained results, the stimulation intensity and stimulation method of the multi-source collaborative stimulation module are regulated to form a closed loop; the closed-loop cognitive impairment evaluation module: based on a heterogeneous graph recurrent neural network to model the dynamic correlation relationship of multi-modal signals, real-time analyzes the cognitive state of patients with SCD / MCI / AD, and dynamically adjusts the stimulation intensity and task difficulty to form a closed-loop link of "signal acquisition → feature fusion → state recognition → parameter iteration".

[0017] As a further improvement of the present invention, the multi-source collaborative stimulation module includes: an auditory stimulation module, a visual stimulation module, a magnetic stimulation module, and an electrical stimulation module.

[0018] As a further improvement of the present invention, the multi-modal physiological signal acquisition module includes a sensor unit for signal sensing and a signal detection unit for signal processing, and can acquire signals such as EEG, ECG, EMG, GSR, and eye movement. The present invention designs a flexible wearable electrocardiogram monitoring system that can meet the requirements of long-term wearing on the basis of meeting the requirements of real-time monitoring. The design of a single-lead electrocardiogram node is completed. Using FPC technology, the entire node is made flexible, further reducing the volume and weight of the node and increasing the flexibility of the node. The final designed flexible electrocardiogram node has a volume of 55mm × 53mm × 5mm and a weight of 6.8g (including the weight of the lithium battery of 4.6g). The communication between the electrocardiogram node and the Android smartphone end monitoring program is realized, and a series of functions such as electrocardiogram waveform display and electrocardiogram data storage are realized. The design of the wearable smart clothing platform is completed.

[0019] As a further improvement of the present invention, in the construction of the multi-modal organ-coupled physiological signal network, due to the complex and close physiological interactions between the brain and the heart during emotional processing, which involve multiple physiological systems. The present invention uses the network physiology method to utilize delay stability as a quantifiable metric to identify and measure the coupling strength between the brain and the heart, especially during visual emotional arousal. The research results reveal the prevalent hemispheric asymmetry in the network connections of the cerebral cortex regions during emotional stimulation, with particular emphasis on the greater interactions exhibited by the right hemisphere. The present invention innovatively proposes an organ-coupled network for fusing multi-modal physiological signals. Using complex network technology, each organ is regarded as a network node, and the coupling between organs is regarded as network edges to construct an emotional-related physiological signal network, which has academic innovation significance and application value for exploring the mechanisms of each organ during the generation and change of emotions.

[0020] Specifically, the multi-modal physiological signal acquisition module includes: a wearable multi-modal physiological signal acquisition module, an organ-intercoupling analysis module based on time-delay stability, and a brain-peripheral physiological signal coupling network construction module.

[0021] Among them, the wearable multi-modal physiological signal acquisition module can collect and process various physiological signals in real time, providing data support for subsequent cognitive impairment assessment and rehabilitation training, including: EEG (electroencephalogram) and EOG (electrooculogram) acquisition modules, ECG (electrocardiogram) and RESP (respiratory signal) acquisition modules, EMG (electromyogram) and PPG (pulse wave) acquisition modules, flexible anhydrous dry electrodes, electrode-skin interface impedance analysis module, adaptive impedance adjustment circuit module, multi-channel low-noise analog front-end module, signal quality assessment module, adaptive filter selection and adjustment module, flexible circuit and hardware comfort improvement module; The EEG (electroencephalogram) and EOG (electrooculogram) acquisition modules: used to monitor electroencephalogram and electrooculogram activities and capture neural signal changes in real time; The ECG (electrocardiogram) and RESP (respiratory signal) acquisition modules: used to monitor electrocardiogram and respiratory activities and obtain dynamic data of physiological states; The EMG (electromyogram) and PPG (pulse wave) acquisition modules: used to detect muscle and pulse activities and obtain relevant information; The flexible anhydrous dry electrodes: used for signal acquisition, improving wearing comfort and reducing skin irritation; The electrode-skin interface impedance analysis module: analyzes the impedance data information between the electrode and the skin and transmits it to the adaptive impedance adjustment circuit module for adjustment; The adaptive impedance adjustment circuit: used to dynamically optimize the electrode impedance, improve signal quality and reduce interference; The multi-channel low-noise analog front-end: used for multi-channel signal acquisition, reducing noise interference and improving signal fidelity; The signal quality assessment module: used to monitor and analyze signal quality in real time to ensure the reliability of data; The adaptive filter selection and adjustment module: used to dynamically select and adjust filters, optimize signal processing and remove noise; The flexible circuit and hardware comfort improvement module: used to improve the comfort and flexibility of device wearing and adapt to different user needs.

[0022] In addition, the inter-organ coupling analysis module and the brain-peripheral physiological signal coupling network construction module based on time-delay stability include: a data segmentation unit, a feature sequence extraction and normalization unit, an inter-organ coupling strength determination unit, and a complex physiological signal network construction unit; the data segmentation unit: performs segmentation processing on the multi-modal physiological signal data collected by the wearable multi-modal physiological signal acquisition module; the feature sequence extraction and normalization unit: extracts the spectral power of 5 frequency bands of EEG in a 2s moving window with a step of 1s: δ (0-4Hz), θ (4-8Hz), α (8-12Hz), σ (12-16Hz), β (16-20Hz); extracts the variance of EOG and EMG signals with a step of 1s in a 2s movement window; the heart rate RR interval and the respiratory interval are both resampled to 1Hz, and then the values are inverted to obtain the heart rate and the respiratory frequency; they are normalized to zero mean and unit standard deviation within an overlapping time window Δt = 60s with a moving step of 30s; synchronous bursts in the normalized signals result in significant cross-correlations being calculated within the window of Δt = 60s and a step of 30s; the inter-organ coupling strength determination unit: the time delay τ0 is determined by the position of the maximum value of the cross-correlation function in each moving window Δt; if the signals corresponding to two systems show a time delay of no more than ±1s in several consecutive time windows, we can determine that the two systems are connected, and the adjacency matrix of the constructed complex network of multi-modal physiological signals is where t ∈ {1, 2,..., T}, and H = W is the number of feature sequence channels; then, different training task states are regarded as different physiological states, and the correlation between the training task states and the physiological network features is studied; the complex physiological signal network construction unit: uses time-delay stability (TDS) to identify and quantify the dynamic connections between physiological systems, and constructs a network reflecting the interactions between physiological systems, where the physiological systems include the brain, heart, respiration, eyes, blood vessels, and muscle activities.

[0023] As a further improvement of the present invention, the stimulation efficacy evaluation module adopts a spatio-temporal attention convolutional network, aiming at the problems of complex early cognitive impairment digital diagnosis and treatment scenarios, large differences in the acquisition quality of multi-modal data, inherent misalignment, and easy loss of modalities and large noise in some modalities in the actual acquisition environment. The present invention innovatively proposes to simultaneously extract temporal features and spatial features based on the spatio-temporal attention convolutional network, and uses the cross-attention mechanism combined with the random modality attenuation / noise addition training technology to reduce the dependence of the model on specific modality features while realizing multi-modal fusion, and improve the robustness of the stimulation efficacy evaluation algorithm. It provides a powerful tool for the diagnosis and treatment of early cognitive impairment, and contributes important experiences and methods to the development of stimulation efficacy evaluation and cognitive load quantification.

[0024] Specifically, the stimulation efficacy evaluation module adopts a spatio-temporal attention convolutional network, including a wearable multi-modal physiological signal acquisition module, a modal cross-attention mechanism, a three-dimensional spatio-temporal representation construction unit, a spatio-temporal convolutional sub-network, and a fusion evaluation module; the wearable multi-modal physiological signal acquisition module: designs multiple cognitive paradigms (episodic memory, executive function, spatial cognition, attentional function, verbal communication, understanding and decision-making, motor balance), and real-time monitors multi-modal physiological data at rest, during stimulation, and in the task state before and after stimulation; measures the resting motor threshold, TMS-evoked potential, and changes in physiological and behavioral characteristics of the corresponding cognitive paradigm before and after stimulation; the modal cross-attention mechanism: improves the evaluation effect of the impact of cognitive function changes under sensory stimulation through a multi-modal fusion method of the modal cross-attention mechanism; the three-dimensional spatio-temporal representation construction unit: constructs a three-dimensional spatio-temporal representation based on multi-modal data, that is, the multi-modal physiological signal spatio-temporal sequence X = (M1, M2,...M T ) ∈ R H×W×T ; the spatio-temporal convolutional sub-network: synchronously extracts the spatial distribution features and time series patterns of physiological signals through a three-dimensional convolutional kernel; the fusion evaluation module: performs stimulation efficacy evaluation after fusing the extracted features.

[0025] Among them, the fusion evaluation module includes: a stimulation efficacy evaluation unit and a cognitive load evaluation unit; the stimulation efficacy evaluation unit: is used to quantitatively evaluate the stimulation effect during the training process, and determines the effectiveness of the stimulation by analyzing physiological and behavioral data; the cognitive load evaluation unit: is used to real-time monitor and evaluate the cognitive load state of the patient during the training process, and identifies changes in cognitive load by analyzing physiological signal characteristics.

[0026] As a further improvement of the present invention, the closed-loop cognitive dysfunction evaluation module. Based on a new understanding of the physiological change law of cognitive function under multi-sensory collaborative stimulation, the present invention explores and discovers several groups of specific parameters that can be real-time collected in a virtual reality environment and are highly correlated with cognitive function, and constructs a new cognitive function evaluation model that fuses multi-physiological parameter behavior information; constructs a new closed-loop adaptive personalized cognitive training plan of "matching cognitive function with multi-sensory collaborative stimulation training paradigm", "real-time feedback of cognitive load - dynamic adjustment of task difficulty", and "matching personality and emotional arousal elements"; at the same time, proposes a heterogeneous graph recurrent neural network to splice the organ-coupled physiological networks constructed at multiple time points in each emotional state into a heterogeneous graph sequence, aggregates features using a graph attention network, learns temporal features through an LSTM, and finally inputs into a GCN and a classification layer for cognitive disorder level evaluation. To more accurately reflect the complexity of emotional and cognitive states. The combination of these strategies will bring a deeper understanding and innovative methods to the field of cognitive evaluation.

[0027] Specifically, the closed-loop cognitive impairment evaluation module uses a heterogeneous graph neural network for the cognitive impairment evaluation method, including: a multi-modal physiological and emotional signal acquisition module, a heterogeneous graph sequence construction, an attention network feature aggregation, a temporal feature learning, and a cognitive impairment level evaluation; the multi-modal physiological and emotional signal acquisition: collecting multi-modal physiological and emotional signals under different emotional states and state transitions, where the signals include various signals collected from different organs; the heterogeneous graph sequence construction: splicing the organ-coupled physiological networks constructed at multiple time points under each emotional state into a heterogeneous graph sequence, and the heterogeneous graph sequence captures the heterogeneity and correlation between modalities; the attention network feature aggregation: using a graph attention network (GAT) to aggregate the features of adjacent nodes of each node to generate a node feature representation containing spatial structure information; the temporal feature learning: learning temporal features from the physiological network representations at different times through a long short-term memory network (LSTM) to obtain a physiological network structure representation containing spatial structure and dynamic evolution information and having discriminative power. The cognitive impairment level evaluation: inputting the physiological network structure representation corresponding to each emotional state into a graph convolutional network (GCN) and a classification layer for cognitive impairment level evaluation.

[0028] A closed-loop evaluation method for cognitive impairment based on the coupling of multiple physiological parameters in multi-source collaborative stimulation, specifically: the multi-modal physiological signal acquisition module collects electroencephalogram, electrocardiogram, and electromyogram physiological signals during the rehabilitation training of cognitive impairment patients under neuromodulation / sensory stimulation through a flexible wearable device, and synchronously constructs a brain-peripheral organ physiological signal coupling network; the stimulation efficacy evaluation module extracts the time-frequency-space features of the physiological signal coupling network based on a spatio-temporal attention network to realize the efficacy evaluation of neuromodulation / sensory stimulation, and dynamically adjusts the neuromodulation / sensory stimulation parameters and training difficulty; the closed-loop cognitive impairment evaluation module models the brain-peripheral organ physiological signal coupling network generated during the rehabilitation training under neuromodulation / sensory stimulation using a heterogeneous graph recurrent network, analyzes the dynamic association between physiological network sequences in the "acquisition-evaluation-regulation" closed-loop link, and realizes the classification of cognitive impairment levels.

[0029] Beneficial effects:

[0030] 1. Improves the comfort and usability of the cognitive rehabilitation training equipment for the elderly, and increases the acceptance and willingness of users.

[0031] 2. Realizes the effective fusion of multi-modal physiological signals, overcomes the modality difference and noise problems, and provides more comprehensive physiological state information.

[0032] 3. Improves the accuracy and robustness of the stimulation efficacy evaluation, and provides a powerful tool for the diagnosis and treatment of early cognitive impairment.

[0033] 4. A personalized cognitive training program with a closed-loop adaptive mechanism is constructed, enabling real-time optimization of cognitive load and dynamic adjustment of task difficulty, thereby enhancing the timeliness of cognitive training and the initiative of participants.

[0034] 5. It promotes the clinical popularization and application of artificial intelligence and digital diagnosis and treatment equipment, improves the research level of cognitive impairment assessment and training in China, and strengthens the independent innovation ability of medical device enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is the project technical route of the present invention.

[0036] Figure 2 This is the wearable multi-modal physiological signal acquisition module of the present invention.

[0037] Figure 3 This is the multi-modal physiological signal acquisition module in the present invention.

[0038] Figure 4 This is the stimulation efficacy evaluation module in the present invention.

[0039] Figure 5 This is the closed-loop cognitive impairment evaluation module in the present invention.

[0040] Figure 6 This is the detailed diagram of the heterogeneous neural network in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0041] The present invention will be further clarified below in conjunction with the drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and not to limit the scope of the present invention.

[0042] Example 1:

[0043] As shown in the figure, the present invention provides a closed-loop cognitive impairment evaluation system based on the coupling of multiple physiological parameters in multi-source collaborative stimulation, including: a multi-modal physiological signal acquisition module 1, a stimulation efficacy evaluation module 2, and a closed-loop cognitive impairment evaluation module 3. The physiological signal acquisition and acquisition module 1 includes a wearable multi-modal physiological signal acquisition device 11, an inter-organ coupling analysis module 12 based on time-delay stability, and a brain-peripheral physiological signal coupling network construction module 13. The multi-modal physiological signal acquisition module 1 can be connected to the inter-organ coupling analysis module 12 based on time-delay stability by wired or wireless means.

[0044] The wearable multi-modal physiological signal acquisition device 11 includes a flexible anhydrous dry electrode unit 111, an electrode-skin interface impedance analysis unit 112, an adaptive impedance adjustment circuit unit 113, a multi-channel low-noise analog front-end unit 114, a signal quality assessment unit 115, an adaptive filtering selection adjustment unit 116, and a flexible circuit and hardware comfort improvement unit 117; the units are integrated into a wearable multi-modal acquisition device to collect EEG\EDG\ECG\RESP\EMG\PPG signals; the multi-source collaborative stimulation module is built into a mobile device or a PC and includes an auditory stimulation unit, a visual stimulation unit, a magnetic stimulation unit, and an electrical stimulation unit;

[0045] The inter-organ coupling analysis module 12 based on time-delay stability includes a data segmentation and feature sequence extraction and normalization unit 121 and an inter-organ coupling strength determination unit 122. The processed data enters the brain-peripheral physiological signal coupling network construction module 13 to construct a physiological signal complex network;

[0046] The data segmentation and feature sequence extraction and normalization unit 121 includes a spectral power unit, an amplitude variance unit, a respiration rate sequence unit, a heart rate sequence unit, an amplitude variance unit, and a pulse rate sequence unit;

[0047] The inter-organ coupling strength determination unit 122 includes a cross-correlation function calculation unit, a practical time-delay parameter extraction unit, a time-delay stability (TDS) judgment unit, and a coupling strength acquisition unit;

[0048] The inter-organ coupling strength determination unit 122 uses time-delay stability (TDS) to identify and quantify the dynamic connections between physiological systems, and constructs a network reflecting the interactions of physiological systems (brain, heart, respiration, eye, blood vessel, and muscle activities). First, the following feature sequences are extracted from the original signals by the feature extraction unit: in a 2s moving window with a step of 1s, the spectral power of 5 frequency bands of EEG is extracted: δ (0-4Hz), θ (4-8Hz), α (8-12Hz), σ (12-16Hz), β (16-20Hz); the variances of EOG and EMG signals in a 2s movement window with a step of 1s are extracted; the heart rate RR interval and the respiration interval are both resampled to 1Hz, and then the values are inverted to obtain the heart rate and the respiration frequency. Therefore, all time series have the same 1s time resolution before analysis; then, the normalization unit normalizes them to zero mean and unit standard deviation within an overlapping time window Δt = 60s with a moving step of 30s. Synchronous bursts in the normalized signals result in significant cross-correlations being calculated within the window of Δt = 60s and a step of 30s. The time delay τ0 is determined by the position of the maximum value of the cross-correlation function in each moving window Δt. In the coupling strength determination unit, if the signals corresponding to two systems show a time delay of no more than ±1s in several consecutive time windows, we can determine that the two systems are connected, and the adjacency matrix of the constructed complex network of multimodal physiological signals is (where t ∈ {1, 2,..., T}, and H = W is the number of channels of the feature sequence); then, different training task states are regarded as different physiological states, and the correlation between the training task states and the physiological network features is studied.

[0049] The stimulus efficacy evaluation module 2 performs emotion recognition and stimulus efficacy evaluation based on a spatio-temporal attention convolutional network, including: a spatio-temporal convolutional network unit 21 and a stimulus efficacy evaluation unit 22;

[0050] The spatio-temporal convolutional network unit 21 includes a closed-loop system of a spatio-temporal attention module, a three-dimensional density connection module, and a transition layer;

[0051] The stimulation efficacy evaluation unit 22 first obtains the data of the person to be measured from the neuromodulation / stimulation process. The neuromodulation / stimulation module 00 designs a variety of cognitive paradigms (episodic memory, executive function, spatial cognition, attention function, verbal communication, understanding decision-making, motor balance), and real-time monitors multi-modal physiological data at rest, during stimulation, and in the task state before and after stimulation, measures the resting motor threshold, TMS-evoked potential, and changes in physiological and behavioral characteristics of the corresponding cognitive paradigm before and after stimulation, and improves the evaluation effect of the impact of cognitive function changes under sensory stimulation through the multi-modal fusion method of the modal cross-attention mechanism. Then, in the three-dimensional spatio-temporal representation construction unit, a three-dimensional spatio-temporal representation (multi-modal physiological signal spatio-temporal sequence X = (M1, M2,...M T ) ∈ R H×W×T ) is constructed according to the multi-modal data. Secondly, in the spatio-temporal convolutional network module, the spatio-temporal convolutional network is used to extract temporal features and spatial features. This network adaptively captures the differential patterns of spatial and time stamps by the attention flow, and at the same time, the attention network can capture local patterns. Finally, in the fusion evaluation unit, the extracted features are fused and then the stimulation efficacy is evaluated. Based on the evaluation results, the stimulation parameters and training intensity are adjusted to achieve the purpose of closed-loop regulation.

[0052] The closed-loop cognitive dysfunction evaluation module 3 includes a graph attention mechanism + LSTM unit 32, a GCN + classification layer unit 33, and a cognitive disorder level classification unit 34;

[0053] The heterogeneous graph recurrent neural network unit 32 includes a graph attention network unit (GAT) 32-1 and a long short-term memory network unit (LSTM) 32-2;

[0054] In the closed-loop cognitive impairment assessment module 3 based on the heterogeneous recurrent neural network, due to the heterogeneity and correlation of multimodal physiological emotion signals in different emotional states and during state transitions. The heterogeneity between modalities is reflected in the differences between the attributes of various signals collected from different organs. The correlation between modalities includes the relationships between channels within the same modality, the relationships between channels of different modalities, and the relationships between the same modality signals before and after different emotional transitions. In the heterogeneous graph recurrent neural network unit 321, the organ-coupled physiological networks constructed at multiple time points in each emotional state are spliced into a heterogeneous graph sequence; then, in the graph attention network unit (GAT) 321-1, the graph attention network is used to aggregate the features of adjacent nodes of each node, and then the long short-term memory network unit (LSTM) 321-2 learns the temporal features from the physiological network representations at different times to obtain a discriminative physiological network structure representation containing spatial structure and dynamic evolution information; finally, in the GCN + classification layer unit 322 and the cognitive impairment level classification module 323, the physiological network structure representation corresponding to each emotional state is input into the GCN and the classification layer for cognitive impairment level assessment.

[0055] The cognitive impairment closed-loop rehabilitation training evaluation system based on multi-physiological parameter monitoring and multi-source collaborative stimulation can be used for real-time cognitive impairment rehabilitation training evaluation, and can be implemented on mobile devices such as mobile phones and tablets, or on the doctor's computer terminal;

[0056] The wearable multi-modal physiological signal acquisition device 11 reads any continuous physiological signals to form physiological signal segments, and the extracted physiological signals are subjected to quality assessment by the signal quality assessment unit to obtain the signals after quality assessment. The closed-loop cognitive dysfunction evaluation module 3 automatically adjusts the stimulation intensity and stimulation method based on the assessment of cognitive load to achieve the most appropriate stimulation effect and obtain the most reasonable cognitive impairment assessment result, so as to better carry out targeted rehabilitation training.

[0057] The present invention also provides a closed-loop evaluation method for cognitive impairment based on the coupling of multiple physiological parameters in multi-source collaborative stimulation, which is as follows: After the multi-modal physiological signals collected by the wearable multi-modal physiological signal acquisition device 11 are processed by the data segmentation and feature sequence extraction normalization unit 121, they then enter the emotion recognition module based on the spatio-temporal attention convolutional network and the cognitive evaluation module based on the heterogeneous recurrent neural network for different data processing. The processing result of the emotion recognition module based on the spatio-temporal attention convolutional network shows the emotion classification of the detected person, and the processing result of the cognitive evaluation module based on the heterogeneous recurrent neural network shows the classification of the cognitive impairment level of the tested person; During the monitoring and analysis process, the stimulation result is fed back to the device generating the stimulation in real time, and according to the analysis of the data, the stimulation parameters and training intensity are adjusted in real time. Finally, the system displays the cognitive impairment detection result on the screen. The entire system forms a closed loop, continuously evaluating the cognitive state and optimizing the intervention strategy.

Claims

1. A closed-loop evaluation system for cognitive impairment based on the coupling of multiple physiological parameters in multi-source synergistic stimulation, characterized in that: include: Multi-source synergistic stimulation module, multi-modal physiological signal acquisition module, stimulation efficacy evaluation module and closed-loop cognitive dysfunction evaluation module; The multi-source collaborative stimulation module: includes a virtual reality-driven multi-sensory stimulation unit and an aging-friendly interactive interface, where the senses include vision, hearing, touch, vestibular, and proprioception, and is used to generate an immersive scene that matches the cognitive training paradigm; The multimodal physiological signal acquisition module: adopts lightweight flexible wearable devices to realize the acquisition of EEG, ECG, EMG, EGG and eye movement signals, optimizes the wearing position and structure weight reduction through ergonomic design, and realizes non-invasive, low-load real-time synchronous acquisition of physiological signals; then, for the collected physiological data, a dynamic correlation network of physiological signals of the brain center and peripheral organs is constructed based on the time-delay stability TDS algorithm, with the brain, heart, breathing, eye and muscle activities as nodes and time-delay correlation as edges, to quantify the network topology of multi-organ synergy; The stimulation effectiveness evaluation module: uses a spatiotemporal attention dense network to synchronously extract the time-frequency-space multi-dimensional features of physiological signals, combines the modal cross-attention mechanism to fuse physiological data and behavioral task performance data, and generates a cognitive load quantitative index; based on the obtained results, the stimulation intensity and stimulation mode of the multi-source collaborative stimulation module are regulated to form a closed loop; The closed-loop cognitive dysfunction assessment module: models the dynamic correlation of multimodal signals based on heterogeneous graph recursive neural networks, analyzes the cognitive state of SCD / MCI / AD patients in real time, dynamically adjusts the stimulation intensity and task difficulty, and forms a closed-loop link of "signal acquisition → feature fusion → state identification → parameter iteration".

2. According to claim 1, a closed-loop evaluation system for cognitive impairment based on coupling of multiple physiological parameters in multi-source collaborative stimulation, characterized in that: The multi-source collaborative stimulation module includes: an auditory stimulation module, a visual stimulation module, a magnetic stimulation module, and an electrical stimulation module.

3. According to claim 1, a closed-loop evaluation system for cognitive impairment based on coupling of multiple physiological parameters in multi-source collaborative stimulation, characterized in that: The multimodal physiological signal acquisition module includes: a wearable multimodal physiological signal acquisition module, an inter-organ coupling analysis module based on time-delay stability, and a brain-peripheral physiological signal coupling network construction module.

4. A closed-loop evaluation system for cognitive impairment based on multi-physiological parameter coupling in multi-source collaborative stimulation according to claim 3, characterized in that: The wearable multimodal physiological signal acquisition module can collect and process a variety of physiological signals in real time, and provide data support for subsequent cognitive impairment assessment and rehabilitation training, including: EEG and EOG electrooculogram acquisition module, ECG and RESP respiratory signal acquisition module, EMG and PPG pulse wave acquisition module, flexible waterless dry electrode, electrode-skin interface impedance analysis module, adaptive impedance adjustment circuit module, multi-channel low-noise analog front-end module, signal quality evaluation module, adaptive filter selection adjustment module, flexible circuit and hardware comfort improvement module; The EEG and EOG acquisition modules are used to monitor brain and eye activity and capture changes in nerve signals in real time. The ECG and RESP respiratory signal acquisition modules are used to monitor cardiac and respiratory activity and obtain dynamic data of physiological status. The EMG electromyography and PPG pulse wave acquisition module is used to detect muscle and pulse activity and obtain relevant information; the flexible waterless dry electrode is used for signal acquisition, improving wearing comfort and reducing skin irritation; The electrode-skin interface impedance analysis module analyzes the impedance data information between the electrode and the skin, and transmits it to the adaptive impedance adjustment circuit module for adjustment; The adaptive impedance adjustment circuit is used to dynamically optimize the electrode impedance, improve signal quality, and reduce interference; The multi-channel low-noise analog front end is used for multi-channel signal acquisition, reducing noise interference and improving signal fidelity; The signal quality assessment module is used to monitor and analyze signal quality in real time to ensure data reliability; The adaptive filter selection and adjustment module is used to dynamically select and adjust filters, optimize signal processing, and remove noise; The flexible circuit and hardware comfort enhancement module are used to improve the comfort and flexibility of wearing the device to meet the needs of different users.

5. The closed-loop evaluation system for cognitive impairment based on multi-physiological parameter coupling in multi-source collaborative stimulation according to claim 3, characterized in that: The inter-organ coupling analysis module based on time-delay stability and the brain-peripheral physiological signal coupling network construction module include: a data segmentation unit, a feature sequence extraction and normalization unit, an inter-organ coupling strength determination unit, and a complex physiological signal network construction unit; The data segmentation unit is configured to segment the multimodal physiological signal data collected by the wearable multimodal physiological signal collection module; The feature sequence extraction and normalization unit: extracts the spectral power of five frequency bands of EEG in a 2s moving window with a step of 1s: δ (0-4Hz), θ (4-8Hz), α (8-12Hz), σ (12-16Hz), β (16-20Hz); extracts the variance of EOG and EMG signals in a 2s moving window with a step of 1s; resamples the heartbeat RR interval and the breathing interval to 1Hz, and then inverts the values ​​to obtain the heart rate and breathing frequency; normalizes to zero mean and unit standard deviation with a moving step of 30s in an overlapping time window Δt = 60s; the synchronous bursts in the normalized signal lead to the calculation of obvious correlations in the window of Δt = 60s and the step of 30s; The inter-organ coupling strength determination unit: The time delay τ0 is determined by the maximum position of the cross-correlation function in each moving window Δt; if the signals corresponding to the two systems show a time delay of no more than ±1s in several consecutive time windows, we can determine that the two systems are connected, and the adjacency matrix of the constructed multimodal physiological signal complex network is Where t∈{1, 2, ..., T}, H=W is the number of feature sequence channels; then, different training task states are taken as different physiological states to study the correlation between the training task states and physiological network characteristics; The complex physiological signal network construction unit uses time-delay stability (TDS) to identify and quantify the dynamic connections between physiological systems and construct a network that reflects the interactions between physiological systems, wherein the physiological systems include brain, heart, breathing, eyes, blood vessels and muscle activities.

6. The closed-loop evaluation system for cognitive impairment based on multi-physiological parameter coupling in multi-source collaborative stimulation according to claim 1, characterized in that: The stimulation effectiveness evaluation module adopts a spatiotemporal attention convolutional network, including a wearable multimodal physiological signal acquisition module, a modality cross-attention mechanism, a three-dimensional spatiotemporal representation construction unit, a spatiotemporal convolutional subnetwork, and a fusion evaluation module; The wearable multimodal physiological signal acquisition module: designs a variety of cognitive paradigms, including episodic memory, executive function, spatial cognition, attention function, verbal communication, comprehension and decision-making, and motor balance, and monitors multimodal physiological data in real time in the resting state before and after stimulation, during stimulation, and in the task state; measures the resting motor threshold before and after stimulation, TMS-evoked potential, and changes in physiological and behavioral characteristics of the corresponding cognitive paradigm; The modal cross-attention mechanism: improves the evaluation effect of the cognitive function change under sensory stimulation through the multimodal fusion method of the modal cross-attention mechanism; The three-dimensional spatiotemporal representation construction unit constructs a three-dimensional spatiotemporal representation according to the multimodal data, i.e., a multimodal physiological signal spatiotemporal sequence X=(M1, M2, ...M T )∈R H×W×T ; The spatiotemporal convolution subnetwork: synchronously extracts the spatial distribution characteristics and time series patterns of physiological signals through a three-dimensional convolution kernel; The fusion evaluation module is used to evaluate the stimulation effectiveness after fusing the extracted features.

7. A closed-loop evaluation system for cognitive impairment based on multi-physiological parameter coupling in multi-source collaborative stimulation according to claim 6, characterized in that: The fusion evaluation module includes: a stimulation effectiveness evaluation unit and a cognitive load evaluation unit; The stimulation effectiveness evaluation unit is used to quantitatively evaluate the stimulation effect during the training process and determine the effectiveness of the stimulation by analyzing physiological and behavioral data; The cognitive load evaluation unit is used to monitor and evaluate the patient's cognitive load status during training in real time, and to identify changes in cognitive load by analyzing physiological signal characteristics.

8. The closed-loop evaluation system for cognitive impairment based on multi-physiological parameter coupling in multi-source collaborative stimulation according to claim 1, characterized in that: The closed-loop cognitive dysfunction evaluation module is based on a heterogeneous graph neural network to perform a cognitive dysfunction evaluation method, including: a multimodal physiological and emotional signal acquisition module, heterogeneous graph sequence construction, attention network feature aggregation, temporal feature learning, and cognitive dysfunction level evaluation; The multimodal physiological and emotional signal collection: collecting multimodal physiological and emotional signals in different emotional states and during state transitions, wherein the physiological and emotional signals include various signals collected from different organs; The heterogeneous graph sequence construction: splicing the organ coupling physiological networks constructed at multiple time points under each emotional state into a heterogeneous graph sequence, wherein the heterogeneous graph sequence captures the heterogeneity and correlation between modalities; The attention network feature aggregation: using the graph attention network GAT to aggregate the features of the adjacent nodes of each node to generate a node feature representation containing spatial structure information; The temporal feature learning: learning the temporal features from the physiological network representations at different times through the long short-term memory network LSTM, and obtaining the physiological network structure representation that contains spatial structure and dynamic evolution information and has discriminative power; The cognitive impairment level assessment: the physiological network structure representation corresponding to each emotional state is input into the graph convolutional network GCN and the classification layer to perform cognitive impairment level assessment.

9. A closed-loop evaluation method for cognitive impairment based on the coupling of multiple physiological parameters in multi-source synergistic stimulation, characterized in that: The system is based on any one of claims 1-8, specifically: a multimodal physiological signal acquisition module acquires EEG, ECG, and EMG physiological signals under neural regulation / sensory stimulation during rehabilitation training of patients with cognitive impairment through flexible wearable devices, and synchronously constructs a brain-peripheral organ physiological signal coupling network; a stimulation effectiveness evaluation module extracts the time-frequency-space characteristics of the physiological signal coupling network based on the spatiotemporal attention network to realize the effectiveness evaluation of neural regulation / sensory stimulation, and dynamically adjusts the neural regulation / sensory stimulation parameters and training difficulty; a closed-loop cognitive dysfunction evaluation module uses a heterogeneous graph recursive network to model the brain-peripheral organ physiological signal coupling network generated by neural regulation / sensory stimulation during rehabilitation training, analyzes the dynamic correlation between physiological network sequences in the "acquisition-evaluation-regulation" closed-loop link, and realizes the classification of cognitive impairment levels.

Images