Cognitive level evaluation system and method applied to caregiver of cerebral infarction patient

Abstract

The invention relates to the technical field of medical auxiliary evaluation and human-computer interaction, in particular to a cognitive level evaluation system and method applied to a caregiver of a cerebral infarction patient. Generating task parameter data based on the multi-modal feature data through a task generation module; the cognitive state decoding module decodes a comprehensive cognitive state by using a double-flow deep learning network to generate a cognitive load, an attention trend and a task response score; the feedback optimization module predicts the change trend of the cognitive ability in combination with a dynamic Bayesian network, and dynamically adjusts task parameters through a closed-loop feedback mechanism; according to the method, the dynamic adaptation of the evaluation task and the design of the intervention task are realized, and the accuracy, interestingness and applicability of evaluation are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical auxiliary assessment and human-computer interaction, and in particular to a cognitive level assessment system and method for caregivers of cerebral infarction patients. Background Art

[0002] Caregivers of patients with cerebral infarction often need to deal with complex decision-making tasks during daily care, such as drug management, emergency treatment, and rehabilitation plan formulation. Accurate assessment of cognitive level is of great significance to improving the coping ability and scientific decision-making ability of caregivers. Traditional assessment methods (Chinese invention patent, publication number: CN113974589B, name: Multimodal behavioral paradigm assessment optimization system and cognitive ability evaluation method) mostly rely on subjective observation or a single cognitive test method, which cannot fully capture the multidimensional cognitive abilities of caregivers, such as attention, memory, and adaptability. Especially in high-stress scenarios, changes in the cognitive state of caregivers often affect the quality of care, and a systematic and accurate assessment method is urgently needed.

[0003] Existing technologies such as the comparative scheme use a multimodal behavioral paradigm to evaluate and optimize the system, combining physiological data and interaction data to model behavior. However, it has the following major defects:

[0004] It focuses on a single physiological or behavioral indicator and lacks deep integration and interactive analysis of multimodal data, making it difficult to form a comprehensive assessment of cognitive status. The existing system lacks a real-time feedback mechanism, and the difficulty and parameter adjustments of the test tasks are relatively simple, and it is impossible to dynamically optimize for different individual characteristics. The form of the test tasks is relatively fixed, which can easily cause fatigue and boredom in the test subjects, thereby affecting the accuracy of the evaluation results. Summary of the invention

[0005] In view of the many problems existing in the above-mentioned prior art, the present invention provides a cognitive level assessment system and method for caregivers of cerebral infarction patients. The present invention obtains EEG, behavior, speech and eye movement data through a multimodal data acquisition module, decodes cognitive status using a dual-stream deep learning network, and predicts the trend of cognitive ability changes in combination with a dynamic Bayesian network. Through the feedback optimization module, personalized intervention tasks are designed and task parameter data are dynamically adjusted to form a closed-loop feedback, thereby achieving accurate assessment and dynamic optimization of cognitive level.

[0006] A cognitive level assessment system for caregivers of patients with cerebral infarction, comprising:

[0007] A multimodal data acquisition module is used to collect EEG data, behavioral data, speech data, and eye movement data, and synchronize and preprocess the collected data to generate multimodal feature data;

[0008] A task generation module, used to generate task parameter data based on the multimodal feature data, wherein the task parameter data includes task type, task difficulty and interference characteristics, and optimize the task parameter data according to the cognitive state of the caregiver to start the assessment task;

[0009] A cognitive state decoding module, used to decode the comprehensive cognitive state of the caregiver based on the multimodal feature data and the task parameter data, and generate comprehensive cognitive state data, wherein the comprehensive cognitive state data includes cognitive load, attention trend and task response score;

[0010] The feedback optimization module is used to design intervention tasks based on comprehensive cognitive state data, adjust the task parameter data for the next stage, and achieve dynamic optimization of assessment tasks and cognitive level assessment through closed-loop feedback.

[0011] Preferably, the multimodal data acquisition module includes an EEG signal acquisition device, a behavior capture device, a voice recording device and an eye tracking device, which are used to respectively collect EEG data, behavior data, voice data and eye movement data; the signals recorded by the EEG signal acquisition device include Delta frequency band signals, Theta frequency band signals, Alpha frequency band signals and Beta frequency band signals; the data recorded by the behavior capture device include mouse trajectory, click position and reaction time; the data collected by the voice recording device include intonation, speech speed and pauses; the data recorded by the eye tracking device include pupil diameter changes and gaze point position.

[0012] Preferably, the task generation module includes a conditional generative adversarial network, which includes a generator and a discriminator. The generator generates task parameter data based on multimodal feature data, and the task parameter data includes task type, task difficulty and interference characteristics; the discriminator is used to verify the generated task parameter data and optimize the generated task parameter data by adjusting the parameters of the generator.

[0013] Preferably, the task generation module further includes a reinforcement learning optimization module, which optimizes the task parameter data based on the task completion status, and the optimization is achieved through a reward function, and the standard form of the reward function is:

[0014]

[0015] Among them, R represents the reward value, C represents the task completion rate, and L represents the cognitive load level.

[0016] Preferably, the cognitive state decoding module includes a dual-stream deep learning network, which is used to decode the comprehensive cognitive state based on multimodal feature data and task parameter data, and the comprehensive cognitive state includes cognitive load, attention trend and task response score; the dual-stream deep learning network includes a first data stream and a second data stream, the first data stream processes the time series characteristics of EEG data and behavioral data through a self-attention mechanism, and the second data stream processes the multimodal characteristics of speech data and eye movement data through a convolutional neural network, and the decoding results of the first data stream and the second data stream are used to generate comprehensive cognitive state data through a feature fusion model.

[0017] Preferably, the cognitive state decoding module further includes a dynamic Bayesian network, which is used to predict the trend of cognitive ability changes based on the comprehensive cognitive state data. The state transition formula of the dynamic Bayesian network is:

[0018]

[0019] Among them, P(S t |S t-1 ) indicates the current state S t In the previous state S t-1 The transition probability under X i represents multimodal input features; P(X i ) represents feature X i The probability distribution of P(S t |S t-1 ,X i ) represents the current state to the previous state and feature X i The conditional probability of .

[0020] Preferably, the feedback optimization module is used to generate intervention tasks based on the comprehensive cognitive state data, and the intervention tasks include attention training tasks, cognitive memory challenge tasks and psychological relaxation tasks; the attention training tasks improve the caregiver's concentration level by adjusting the number of visual targets and the target switching speed, the cognitive memory challenge tasks improve the caregiver's cognitive load tolerance by increasing the complexity of task interference elements, and the psychological relaxation tasks relieve the caregiver's decision-making pressure by controlling the deep breathing guidance rhythm and audio relaxation frequency.

[0021] Preferably, the feedback optimization module further includes an intervention effect analysis unit, and the data collected by the intervention effect analysis unit includes intervention completion rate data, real-time physiological response data and behavior adjustment data; the comprehensive cognitive state data before and after the intervention are compared and analyzed through a multimodal regression analysis model to generate intervention effect evaluation data, and the intervention effect evaluation data is used to optimize the task parameter data for the next stage.

[0022] Preferably, the feedback optimization module transmits the intervention effect evaluation data and the adjusted next stage task parameter data to the task generation module through a closed-loop feedback mechanism, which is used to dynamically adjust the task type, task difficulty and interference characteristics of subsequent evaluation tasks, thereby achieving dynamic adaptation of the evaluation tasks to the caregiver's cognitive state.

[0023] A method for executing the cognitive level assessment system for caregivers of cerebral infarction patients comprises the following steps:

[0024] Collect EEG data, behavioral data, speech data and eye movement data through a multimodal data acquisition module; perform frequency band decomposition on the EEG data to extract Delta band, Theta band, Alpha band and Beta band signals; perform time synchronization and outlier removal on the behavioral data; perform filtering and denoising on the speech data to extract intonation, speech speed and pause features; perform Gaussian smoothing on the eye movement data to extract pupil diameter changes and gaze point position information; align the above processed data through a time synchronization algorithm to generate multimodal feature data;

[0025] Based on the multimodal feature data, task parameter data is generated through a task generation module, and the task parameter data includes task type, task difficulty and interference characteristics; the task parameters are generated by using a generator in a conditional generative adversarial network, and the generated task parameters are verified and optimized by a discriminator; according to the task execution status, the task parameter data is optimized in real time through a reinforcement learning optimization module, and a reward function is used to balance and optimize the task completion rate and cognitive load level, and the reward function is:

[0026]

[0027] Among them, R represents the reward value, C represents the task completion rate, and L represents the cognitive load level;

[0028] Based on the multimodal feature data and task parameter data, a comprehensive cognitive state is extracted through a dual-stream deep learning network in a cognitive state decoding module, wherein the comprehensive cognitive state includes cognitive load, attention trend and task response score; the first data stream is used to process the time series features of EEG data and behavioral data, and the second data stream is used to process the multimodal features of speech data and eye movement data; the decoding results of the first data stream and the second data stream are used to generate comprehensive cognitive state data through a feature fusion model; a dynamic Bayesian network is used to perform state transition modeling based on the comprehensive cognitive state data to predict future cognitive ability change trends, and the state transition modeling formula is:

[0029]

[0030] Among them, P(S t |St-1 ) indicates the current state S t In the previous state S t-1 The transition probability under X i represents multimodal input features; P(X i ) represents feature X i The probability distribution of P(S t |S t-1 ,X i ) represents the current state to the previous state and feature X i The conditional probability of

[0031] Intervention tasks are designed based on comprehensive cognitive state data, and the intervention tasks include attention training tasks, cognitive memory challenge tasks, and psychological relaxation tasks; intervention task parameters are adjusted, including the number of visual targets, the complexity of task interference elements, and the rhythm of deep breathing guidance; feedback data after the completion of the intervention task is collected, including intervention completion rate data, physiological response data, and behavior adjustment data, and intervention effect evaluation data is generated using a multimodal regression analysis model; based on the intervention effect evaluation data and the adjusted task parameter data, the task parameters of the next stage are optimized through a closed-loop feedback mechanism, and the task type, task difficulty, and interference characteristics of subsequent tasks are dynamically adjusted to achieve dynamic adaptation of the evaluation task to the cognitive state of the caregiver.

[0032] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0033] The system of the present invention combines EEG data, behavioral data, speech data and eye movement data, and uses a deep learning model to decode the comprehensive cognitive state to achieve multi-dimensional cognitive level assessment;

[0034] The present invention optimizes the task parameters of the next stage based on the intervention effect evaluation data through a closed-loop feedback mechanism, thereby achieving dynamic adaptation of the evaluation task to the individual status;

[0035] The present invention adopts generative adversarial networks and reinforcement learning optimization modules to generate personalized task parameter data, improve the diversity and fun of task forms, and effectively alleviate test fatigue;

[0036] The present invention provides a scientific cognitive ability assessment method for caregivers of cerebral infarction patients, which can assist in the formulation of personalized nursing plans and improve the care effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a structural block diagram of the system of the present invention;

[0038] Figure 2 It is the internal logic block diagram of the task generation module in the present invention;

[0039] Figure 3It is a closed-loop feedback optimization process flow chart of the present invention;

[0040] Figure 4 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0041] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure.

[0042] like Figure 1 As shown, a cognitive level assessment system for caregivers of patients with cerebral infarction comprises:

[0043] A multimodal data acquisition module is used to collect EEG data, behavioral data, speech data, and eye movement data, and synchronize and preprocess the collected data to generate multimodal feature data;

[0044] Preferably, the multimodal data acquisition module includes an EEG signal acquisition device, a behavior capture device, a voice recording device and an eye tracking device, which are used to respectively collect EEG data, behavior data, voice data and eye movement data; the signals recorded by the EEG signal acquisition device include Delta frequency band signals, Theta frequency band signals, Alpha frequency band signals and Beta frequency band signals; the data recorded by the behavior capture device include mouse trajectory, click position and reaction time; the data collected by the voice recording device include intonation, speech speed and pauses; the data recorded by the eye tracking device include pupil diameter changes and gaze point position.

[0045] The main function of the multimodal data acquisition module is to collect comprehensive data from the caregiver's various behaviors and physiological responses to support subsequent cognitive level analysis and intervention optimization. The module includes an EEG signal acquisition device, a behavior capture device, a voice recording device, and an eye tracking device, which are used to collect EEG data, behavior data, voice data, and eye movement data, and synchronize and preprocess the data to generate multimodal feature data for analysis. The following is a detailed description of the specific principles and application effects of this module:

[0046] The EEG signal acquisition device is used to record the EEG activity of the caregiver, especially the key frequency band signals related to cognitive state and emotion, including Delta (0.5-4Hz), Theta (4-8Hz), Alpha (8-13Hz) and Beta (13-30Hz) bands. These frequency bands reflect cognitive functions such as attention, relaxation state, working memory load, etc. By acquiring and preliminarily processing the signals of these frequency bands (such as filtering denoising and artifact removal), key features related to the current cognitive load of the caregiver can be extracted. For example, during the assessment process, the enhancement of the Delta frequency band signal may indicate a fatigue state, while the enhancement of the Beta frequency band signal may reflect a high cognitive load. In specific implementation, the acquisition device arranges electrodes in the forehead, occipital lobe and parietal lobe of the caregiver through a head-mounted EEG recording device, records electrical signals in real time, and combines bandpass filtering technology to remove power frequency noise and artifacts to ensure that the generated EEG feature data has a high signal-to-noise ratio and structured features.

[0047] Behavior capture devices are used to record the caregiver's operational behavior data, including mouse trajectories, click locations, and reaction times. These data reflect the caregiver's task completion mode, reaction speed, and operational stability, and are important dimensions for assessing their cognitive state. For example, the complexity of the mouse trajectory can reflect the caregiver's behavioral planning ability and concentration when completing the task, and the reaction time is directly related to the decision-making efficiency. In actual collection, the behavior capture device is connected to the interactive recording interface of the evaluation platform to record the caregiver's operational behavior during the task execution in real time, and eliminate abnormal data points (such as too short click time or operation deviation) through algorithms to ensure the accuracy and validity of the data.

[0048] The voice recording device collects the caregiver's voice signals, including features such as intonation, speaking speed and pauses, to analyze the caregiver's emotional state and language cognitive load. In the care of patients with cerebral infarction, voice features can be used to detect whether the caregiver has emotional fluctuations, anxiety or excessive stress. For example, large changes in intonation or significantly faster speaking speed may indicate a high stress state, while frequent pauses may reflect excessive cognitive load or slow thinking. In specific implementation, the voice recording device captures the voice signal through a high-sensitivity microphone, and combines the short-time Fourier transform technology to decompose the frequency domain features of the voice signal, extract the dynamic change trends of pitch and speaking speed, and form structured voice feature data.

[0049] The eye tracking device is used to record the caregiver's gaze point position and pupil diameter changes, which can reflect their visual attention allocation and psychological workload. For example, looking at the same position for a long time may indicate focused attention, while frequent gaze shifts may reflect distraction or difficulty in task understanding. At the same time, the expansion of pupil diameter is often associated with high cognitive load or emotional arousal. Eye movement data is collected through a high-frame rate eye tracking camera, combined with an optical tracking algorithm to determine the pupil position and its dynamic changes, and Gaussian smoothing is used to remove fluctuations caused by changes in ambient light.

[0050] In summary, the multimodal data acquisition module captures the behavior and physiological characteristics of the caregiver from multiple dimensions through the collaborative work of the EEG signal acquisition device, behavior capture equipment, voice recording device and eye tracking device, generating synchronized and high-quality multimodal feature data. These data provide a comprehensive basis for the accurate decoding and dynamic evaluation of cognitive state, ensuring that the system can accurately reflect the actual cognitive level and stress state of the caregiver. At the same time, the design of the module fully considers the characteristics of the care task of patients with cerebral infarction, such as the diversity of the task environment and the fluctuation of the caregiver's psychological state, so that the system is robust and efficient in practical applications.

[0051] like Figure 2 As shown, a task generation module is used to generate task parameter data based on the multimodal feature data, wherein the task parameter data includes task type, task difficulty and interference characteristics, and optimize the task parameter data according to the cognitive state of the caregiver to start the assessment task;

[0052] The core function of the task generation module is to generate task parameter data based on multimodal feature data, including task type, task difficulty, and interference characteristics, and optimize task parameters in combination with the caregiver's cognitive state to start the assessment task. Through this module, personalized tasks that match the caregiver's cognitive ability can be dynamically created to ensure the accuracy and pertinence of the assessment process, while providing the necessary data support for subsequent analysis and feedback optimization.

[0053] The implementation of the task generation module is based on the conditional generative adversarial network (cGAN), which consists of two core parts: the generator and the discriminator. The generator receives multimodal feature data input, including EEG frequency band feature data, behavioral feature distribution data, voice emotion transfer data, and eye movement feature data, and generates task parameter data. The task parameter data covers three aspects: task type (such as visual memory, logical reasoning, etc.), task difficulty (such as the number of targets, task duration), and interference characteristics (such as background visual interference, sound interference). The discriminator verifies the generated task parameter data, and optimizes the parameters of the generator by comparing the correlation between the generated task and the input multimodal feature data to ensure that the generated task parameters are highly matched with the current state of the caregiver. For example, for caregivers with high cognitive load, the generator may generate simple single-task types and task parameters with low difficulty, while the discriminator further adjusts the task interference characteristics through a feedback mechanism to minimize interference to ensure the completeability of the task.

[0054] Another key part of the module is the reinforcement learning optimization module, which is mainly used to dynamically adjust task parameters according to the caregiver's performance during task execution. By dynamically calculating the relationship between task completion rate and cognitive load level, the system is able to automatically adjust task difficulty and interference characteristics. For example, when the caregiver shows a high completion rate in the task but with a high cognitive load level, the system will appropriately reduce the difficulty of the next stage of the task or reduce interference factors to balance the challenge and operability of the task.

[0055] The application effect of the task generation module is that it can achieve dynamic adaptation of task parameters, thereby enhancing the pertinence and personalization of the assessment. Specifically, the generated task parameter data directly affects the caregiver's participation and performance stability in the assessment task. For example, by generating memory tasks with clear visual targets and less background interference for caregivers with low concentration, their participation and assessment accuracy can be effectively improved. At the same time, the task parameter data generated by this module also provides important input data for the cognitive state decoding module, further improving the accuracy of the comprehensive cognitive state analysis.

[0056] Taking the specific implementation as an example, in a visual memory assessment, the multimodal feature data of a caregiver of a patient with cerebral infarction showed a high Delta frequency band signal and complex behavioral trajectory, indicating a high cognitive load and distracted attention. In this case, the task generation module generated task parameters with the task type of visual target recognition, low task difficulty (the number of targets was 2, and the time limit was 30 seconds), and interference characteristics of no dynamic background. After the caregiver completed the task, the system analyzed the task completion rate and cognitive load level through the reinforcement learning optimization module, and appropriately increased the difficulty of the next task (the number of targets was increased to 3) to increase the challenge of the assessment. This process reflects the core role of the task generation module in the design of dynamic adaptive tasks.

[0057] In summary, the task generation module can generate task parameter data that is adapted to the caregiver's cognitive state based on multimodal feature data through conditional generative adversarial networks and reinforcement learning optimization mechanisms, and dynamically adjust the task type, difficulty, and interference characteristics to achieve accurate matching and personalized design of assessment tasks. The implementation of this module not only ensures the flexibility of the assessment task, but also provides reliable input data for accurate decoding of cognitive states, while optimizing the caregiver's task experience and assessment effect through dynamic task adjustment.

[0058] Preferably, the task generation module includes a conditional generative adversarial network, which includes a generator and a discriminator. The generator generates task parameter data based on multimodal feature data, and the task parameter data includes task type, task difficulty and interference characteristics; the discriminator is used to verify the generated task parameter data and optimize the generated task parameter data by adjusting the parameters of the generator.

[0059] The task generation module generates and optimizes task parameter data through conditional generative adversarial networks (cGANs), providing dynamic and personalized task design for the cognitive level assessment of caregivers of patients with cerebral infarction. The conditional generative adversarial network consists of two parts: a generator and a discriminator. The generator takes multimodal feature data as input to generate task parameter data; the discriminator verifies the generated task parameter data and optimizes the parameters of the generator through a feedback mechanism to ensure that the generated task parameters are adapted to the cognitive state of the caregiver. The following describes the function and implementation of this module in detail from the perspective of principle and application effect.

[0060] The role of the generator is to generate task parameter data based on multimodal feature data, which includes task type, task difficulty and interference characteristics. Multimodal feature data includes EEG frequency band characteristics, behavioral response characteristics, voice emotion characteristics and eye movement characteristics, which are highly correlated with the cognitive state of the caregiver. For example, the enhancement of Beta signals in the EEG frequency band may indicate a higher cognitive load, while the complexity of trajectories in behavioral response data may indicate a distracted state. The generator constructs a conditional generation model through a deep neural network, and uses multimodal feature data as conditional input to generate targeted task parameter data. Task types can be visual memory, logical reasoning or attention allocation, etc.; task difficulty is reflected by the number of targets, complexity of operation steps or time limit; interference characteristics include visual background dynamics, the ambiguity of task targets or external audio interference. For example, for distracted caregivers, the generator can generate task parameters with visual recognition as the task type, a small number of targets and low dynamic interference.

[0061] The function of the discriminator is to verify whether the task parameter data generated by the generator matches the input multimodal feature data, and optimize the generator through a feedback mechanism. The discriminator adopts a binary classification neural network architecture, with the input being the generated task parameter data and multimodal feature data, evaluates the correlation between the two, and outputs the credibility score of the task parameter. When the generated task parameter data is highly matched with the multimodal feature data, the discriminator gives a high score; otherwise, the parameters of the generator are adjusted through error feedback to improve the task generation quality of the generator. The verification process of the discriminator ensures that the generated task parameter data not only meets the task design requirements, but also truly reflects the current cognitive state of the caregiver. For example, when the caregiver's multimodal feature data indicates a high cognitive load, the discriminator will tend to accept low-difficulty, low-interference task parameter data, and reject high-difficulty tasks that do not match the feature data.

[0062] The application effect of the conditional generative adversarial network is reflected in the dynamic adaptability and accuracy of task design. Through the collaborative work of the generator and the discriminator, the task generation module can generate task parameters that are highly matched with the cognitive state of the caregiver in real time, optimize the evaluation experience and improve the success rate of task completion. For example, when a caregiver's EEG characteristics show high Beta frequency band signals and behavioral characteristics show disordered mouse trajectories, the task generation module generates a simple visual memory task with 2 targets and no background interference; the discriminator verifies the task parameters and believes that they match the current state, and the task is adopted. Through this task, the system can collect evaluation data while avoiding affecting the caregiver's participation due to the complexity of the task.

[0063] In the specific implementation, the parameter optimization of the generator depends on the feedback signal of the discriminator. The training process of the discriminator is completed by comparing the real task parameter data with the generated task parameter data. The generator uses conditional input to construct the initial task parameter data. The discriminator verifies the quality of the data through supervised learning and gradually optimizes the weight of the generator. The whole process realizes the adaptive adjustment of task parameter design, ensuring that each task matches the state of the caregiver, thereby providing high-quality data input for subsequent cognitive state analysis and intervention.

[0064] In summary, the task generation module achieves the dynamics and accuracy of task design through conditional generative adversarial networks. The generator generates task parameter data based on multimodal feature data, and the discriminator optimizes the output of the generator by checking to ensure that the generated task parameters are highly consistent with the cognitive state of the caregiver. This module not only improves the adaptability of the assessment task, but also enhances the system's task design capabilities in complex scenarios, providing scientific and efficient technical support for the cognitive level assessment of caregivers of patients with cerebral infarction.

[0065] Preferably, the task generation module further includes a reinforcement learning optimization module, which optimizes the task parameter data based on the task completion status, and the optimization is achieved through a reward function, and the standard form of the reward function is:

[0066]

[0067] Among them, R represents the reward value, C represents the task completion rate, and L represents the cognitive load level.

[0068] The reinforcement learning optimization module in the task generation module is a core component that dynamically adjusts task parameters. It is used to optimize task parameter data based on the caregiver's task completion status, and adaptively improve the adaptability and personalization of the assessment task through the reinforcement learning mechanism. Its key principle is to construct a reward function with task completion rate and cognitive load level as core indicators, and dynamically adjust task parameters by optimizing the reward value, thereby balancing the task difficulty and the caregiver's cognitive ability. The following will explain in detail the function and implementation of this module from the perspective of principle, effect and implementation.

[0069] The core of the reinforcement learning optimization module is to adjust the task parameter data based on the caregiver's real-time task completion. The optimization process is guided by the reward function, and the standard form of the reward function is shown above.

[0070] The task completion rate C is a key indicator of the caregiver's completion of the assessment task, which is usually measured by calculating the number or accuracy of task completion targets. For example, in a visual recognition task, the completion rate can be expressed as the ratio of the number of correctly identified targets to the total number of targets. The cognitive load level L is an important parameter that reflects the current state of the caregiver and can be indirectly evaluated by the frequency band characteristics of the EEG signal (such as Beta band enhancement) or behavioral characteristics (such as mouse trajectory complexity). Through the reward function R, the reinforcement learning optimization module can achieve a dynamic balance between task completion rate and cognitive load, avoiding excessive fatigue of the caregiver due to tasks of excessive difficulty, while ensuring that the task is sufficiently challenging.

[0071] In the specific implementation, the reinforcement learning optimization module realizes the dynamic adjustment of task parameters through a policy optimization algorithm (such as deep Q network or proximal policy optimization). The input of the module is the completion rate and cognitive load level of the caregiver in the current task, and the output is the adjusted task parameter data, including task type, task difficulty and interference characteristics. For example, in a logical reasoning task, if the caregiver's task completion rate is high (such as 90%) but the cognitive load level is too high (such as a significant increase in the Beta band signal), the optimization module may reduce the difficulty of the next task by reducing the number or complexity of logical problems and adjusting the interference characteristics to reduce cognitive load.

[0072] The effects of the reinforcement learning optimization module are reflected in the following aspects: First, it can dynamically adapt the difficulty of the task so that the task design is highly consistent with the caregiver's current cognitive state, thereby improving the effectiveness of the assessment. For example, in a continuous attention allocation task, the optimization module can adjust the number or time interval of task goals by real-time monitoring of the completion rate and cognitive load to ensure that the caregiver can complete the task with moderate challenges. Secondly, this module significantly improves the level of personalization of the assessment task. Through adaptive adjustment of task parameters, caregivers with different cognitive abilities can achieve the best performance in the assessment without affecting the accuracy of the assessment due to tasks that are too difficult or too easy.

[0073] Taking the specific implementation as an example, a caregiver of a patient with cerebral infarction showed a high task completion rate (90%) and a moderate cognitive load level (mild enhancement of Beta band signal) in a dynamic visual target recognition task. The reinforcement learning optimization module calculated a higher reward value based on the reward function, indicating that the current task parameter design is more suitable. Therefore, in the next task, the number of targets is only slightly increased (from 5 to 6) to slightly increase the task difficulty, and the dynamic interference of the visual background is reduced to reduce cognitive load. Through this optimization strategy, the system further improved the completion rate of the caregiver in subsequent tasks and effectively controlled its cognitive load level.

[0074] In summary, the reinforcement learning optimization module dynamically adjusts the task design to adapt to the cognitive state of the caregiver through the task parameter optimization mechanism based on the reward function, ensuring the adaptability and personalization of the task. The implementation of the module combines the real-time data input of the task completion rate and cognitive load level, and adjusts the task parameters through the strategy optimization algorithm, which effectively improves the accuracy and flexibility of the assessment task, and provides intelligent support for the cognitive level assessment of the caregivers of cerebral infarction patients.

[0075] A cognitive state decoding module, used to decode the comprehensive cognitive state of the caregiver based on the multimodal feature data and the task parameter data, and generate comprehensive cognitive state data, wherein the comprehensive cognitive state data includes cognitive load, attention trend and task response score;

[0076] The cognitive state decoding module uses a multimodal deep learning network to decode and analyze multimodal feature data and task parameter data to extract key information that can reflect the comprehensive cognitive state of the caregiver. Multimodal feature data include frequency band characteristics, behavioral response characteristics, voice emotion characteristics, and eye movement trajectory characteristics of EEG signals, which are closely related to the caregiver's attention allocation, cognitive load, and task responsiveness. Task parameter data provides contextual information of the current assessment task, including task type, difficulty, and interference characteristics, which is used to provide background reference for the decoding process. By combining multimodal data with task parameters, the module can generate more accurate cognitive state decoding results.

[0077] The core algorithm of the module adopts a dual-stream deep learning network, which includes two decoding data streams: a time series data stream and a multimodal feature fusion stream. The time series data stream is responsible for decoding the time-dependent features in EEG signals and behavioral response data, and uses the long short-term memory network (LSTM) to extract the dynamic changes of EEG frequency bands over time and the complexity and change trend of the trajectory in the behavioral data. The multimodal feature fusion stream focuses on voice emotions and eye movement features, and extracts the pattern features of the emotion transfer matrix and the gaze point trajectory through the convolutional neural network (CNN). The results of the two data streams are weighted and combined through the feature fusion module to generate comprehensive cognitive state data.

[0078] For example, when evaluating the attention trend of the caregiver, the time series data stream analyzes the change trend of the Alpha frequency band in the EEG signal, and combines the stability of the mouse trajectory in the behavioral data to draw a preliminary conclusion about the concentration of attention; the multimodal feature fusion stream further analyzes the stability of the gaze point of the eye movement trajectory and the stability of the tone in the voice to comprehensively derive the change trend of attention. Finally, the feature fusion module weights the above results and outputs them to generate an accurate score that reflects the attention trend.

[0079] The comprehensive cognitive state data generated by the cognitive state decoding module plays multiple roles in assessing the cognitive level of caregivers. First, cognitive load data quantifies the psychological stress and resource consumption of caregivers during task execution. For example, the presence of high Beta frequency band signals usually indicates high cognitive load. Second, attention trend data provides a dynamic assessment of the caregiver's attention allocation during the task. By analyzing the frequency of gaze point offset and the fluctuation of voice emotion, the system can accurately capture the peak and trough periods of attention. The task response score reflects the caregiver's overall performance in the task, including the accuracy, reaction time and stability of task completion. These indicators together constitute a comprehensive evaluation of the caregiver's execution ability.

[0080] Taking the specific implementation as an example, a caregiver of a cerebral infarction patient is performing a visual memory task. The task type is to identify key targets in the picture, with a medium difficulty (the number of targets is 5) and no background interference. The cognitive state decoding module collects the following data during the task execution: the power changes of the Alpha and Beta bands of the EEG signal, the offset amplitude of the mouse trajectory, the emotional transfer matrix in the voice, and the distribution of the gaze points on the screen. The time series data stream extracts the characteristics of the rapid decline of the power of the Alpha band at the beginning of the task, and combined with the stability of the mouse trajectory, it is concluded that the concentration is low; the multimodal feature fusion stream analyzes the distribution of the gaze points and finds that they frequently deviate from the target area, which further verifies the state of distraction. Based on the above results, the module outputs an attention trend score of 45 (out of 100), and a cognitive load score of 80, indicating that the caregiver is in a high cognitive load state, while the task response score is 60, reflecting a medium level of task completion.

[0081] The cognitive state decoding module generates key data reflecting the comprehensive cognitive state of the caregiver through deep decoding of multimodal feature data and task parameter data, supporting the scientificity and accuracy of the assessment process. Its dual-stream deep learning network architecture achieves efficient fusion of time series features and multimodal features, laying the foundation for the accurate generation of cognitive load, attention trends and task response scores. The application of the module significantly improves the flexibility and pertinence of the assessment tasks, and provides strong technical support for the assessment of the cognitive level of caregivers of patients with cerebral infarction.

[0082] Preferably, the cognitive state decoding module includes a dual-stream deep learning network, which is used to decode the comprehensive cognitive state based on multimodal feature data and task parameter data, and the comprehensive cognitive state includes cognitive load, attention trend and task response score; the dual-stream deep learning network includes a first data stream and a second data stream, the first data stream processes the time series characteristics of EEG data and behavioral data through a self-attention mechanism, and the second data stream processes the multimodal characteristics of speech data and eye movement data through a convolutional neural network, and the decoding results of the first data stream and the second data stream are used to generate comprehensive cognitive state data through a feature fusion model.

[0083] The two-stream deep learning network consists of a first data stream and a second data stream, which are used to process time series features and multimodal features, respectively. The first data stream focuses on the time series analysis of EEG data and behavioral data, and captures the key time points and change trends in the signal through the self-attention mechanism. In EEG data, changes in the Alpha band are usually related to attention, while the Beta band reflects the level of cognitive load. Trajectory complexity and reaction time in behavioral data also provide important information about the task completion status. The self-attention mechanism generates a time series feature representation by weighted aggregation of feature values ​​at different time points in the time series, highlighting data changes at key moments. For example, in a task, the enhancement of the Beta band that occurs in the middle of the task for the caregiver may be a key point of high cognitive load, and the self-attention mechanism will weight this feature to ensure that the model pays attention to this important information.

[0084] The second data stream focuses on multimodal feature processing of speech data and eye movement data, and extracts spatial pattern features through convolutional neural networks (CNN). In speech data, intonation, speech rate, and pause features can reflect the emotional state and language load of the caregiver; the distribution of gaze points and changes in pupil diameter in eye movement data are important indicators of attention allocation and psychological load. Convolutional neural networks extract local patterns of these features through convolution operations. For example, the dynamic change pattern of eye movement trajectories can reveal the caregiver's attention shifting tendency during the task, while the pause features in speech emotions can indicate changes in their cognitive stress.

[0085] The decoding results of the first data stream and the second data stream are weighted and combined through the feature fusion model to generate comprehensive cognitive state data. The feature fusion model uses the attention mechanism and weighting strategy to uniformly map the results of the two data streams to the comprehensive cognitive state indicator space, including cognitive load, attention trend, and task response score. For example, the EEG Beta band features from the first data stream and the gaze point distribution features from the second data stream may jointly indicate the high cognitive load state of the caregiver. The fusion model weights these results according to the feature weights and finally generates an accurate cognitive load score.

[0086] The dual-stream deep learning network of the cognitive state decoding module can fully reflect the cognitive state characteristics of the caregiver. By analyzing the intensity of the EEG Beta band signal, the complexity of the mouse trajectory, and the dynamic changes in the pupil diameter, the system can quantify the psychological pressure and resource consumption of the caregiver in task execution. For example, when the cognitive load score is high, the system can adjust the difficulty of subsequent tasks to reduce the pressure on the caregiver. Combined with the stability of the gaze point distribution, the fluctuation of the voice tone, and the change trend of the Alpha band signal, the system can dynamically capture the concentration and dispersion of the caregiver's attention. For example, in the case of decreased attention, the system can design more intuitive and less distracting tasks to improve the level of attention. By comprehensively analyzing the task completion rate, reaction time, and the stability of the behavior trajectory, the system can evaluate the overall performance of the caregiver in the task and provide an objective score for the task completion status.

[0087] Take a visual search task performed by a caregiver of a cerebral infarction patient as an example. The task parameters are 5 targets and no dynamic interference background. The first data stream extracts the Alpha and Beta band features from the EEG signal and finds that the power of the Alpha band is high at the beginning of the task, indicating focused attention, but the Beta band signal is significantly enhanced in the middle of the task, indicating an increase in cognitive load. At the same time, the behavioral data shows that the mouse trajectory begins to shift. The second data stream extracts the gaze point distribution features from the eye movement data and finds that frequent switching of gaze points indicates distracted attention, while the intonation analysis in the voice data shows a decrease in emotional stability. The feature fusion model combines the above information to generate a cognitive load score of 85 (high load state), an attention trend score of 50 (low concentration), and a task response score of 70 (medium completion quality). Based on these results, the system can adjust the parameters of the next task to reduce the number of targets and reduce interference to improve the caregiver's task performance.

[0088] The cognitive state decoding module achieves accurate decoding of multimodal features through a dual-stream deep learning network. Its self-attention mechanism and convolutional neural network extract key information from time series and spatial features respectively, and the feature fusion model integrates the results of different data streams into comprehensive cognitive state data. The application of the module not only improves the decoding accuracy and efficiency of the system, but also provides scientific support for task design and intervention optimization, significantly enhancing the pertinence and practicality of cognitive level assessment for caregivers of patients with cerebral infarction.

[0089] Preferably, the cognitive state decoding module further includes a dynamic Bayesian network, which is used to predict the trend of cognitive ability changes based on the comprehensive cognitive state data. The state transition formula of the dynamic Bayesian network is:

[0090]

[0091] Among them, P(S t |S t-1 ) indicates the current state S t In the previous state S t-1 The transition probability under X i represents multimodal input features; P(X i ) represents feature X i The probability distribution of P(S t |S t-1 ,X i ) represents the current state to the previous state and feature X i The conditional probability of .

[0092] Dynamic Bayesian Network (DBN) is a time series probability model used to describe the conditional probability relationship between states in a time series.

[0093] Through the above formula, the dynamic Bayesian network combines comprehensive cognitive state data (such as cognitive load, attention trend and task response score) and multimodal feature data (such as EEG signals, behavioral trajectories, voice emotions and eye movement characteristics) to predict the temporal change trend of cognitive state.

[0094] Specifically, the dynamic Bayesian network decomposes the comprehensive cognitive state into multiple sub-states (such as the current cognitive load level and the degree of concentration), and uses the time series characteristics of multimodal feature data to establish the transition relationship between different states. For example, the enhancement of the Beta frequency band in the EEG signal may indicate the continuation of the high cognitive load state; the increase in intonation fluctuations in speech features may reflect the changing trend of cognitive stress. By mapping these input features to the state transition model, the dynamic Bayesian network can infer the correlation between the current cognitive state and the future state.

[0095] Through time-series analysis of comprehensive cognitive state data, the model can predict future cognitive load levels, attention trends, and task response performance. For example, after a period of high-intensity tasks, if the state transition model predicts that cognitive load may continue to rise, the system can design low-load tasks in advance to reduce stress.

[0096] The dynamic Bayesian network provides decision support for the design of subsequent tasks. For example, by predicting the decline in attention trend, the system can adjust task parameters to improve task completion rate.

[0097] By modeling multimodal feature data with a probabilistic model, the dynamic Bayesian network can effectively handle the noise and uncertainty in the input data and enhance the robustness of the prediction results.

[0098] Taking the dynamic assessment of a caregiver of a patient with cerebral infarction as an example, the comprehensive cognitive state data includes the cognitive load score of the current task (70 points), the attention trend score (60 points), and the task response score (75 points). The multimodal feature data comes from the Beta band power of the EEG signal, the trajectory deviation of the behavioral trajectory, and the pupil diameter change in the eye movement feature. The state transition model of the dynamic Bayesian network makes predictions based on these data:

[0099] The model is expressed by the formula P(S t |S t-1 ,X i ) Calculate the transition probability from the current cognitive state to the next moment.

[0100] The significant increase in Beta frequency power in EEG data leads to high P(S t |S t-1 ,X i ) value, indicating that cognitive load has a continuous upward trend.

[0101] The possibility of decreased attention was further supported by the increased trajectory deviation in the behavioral data.

[0102] The comprehensive model output predicted results: in the future, cognitive load may rise to 85 points, attention trend may drop to 50 points, and task response score may drop to 65 points.

[0103] Based on the above predictions, the system can reduce the difficulty of the task (reduce the number of targets) and reduce interference characteristics (reduce the dynamics of the visual background) in the next stage of the task to reduce cognitive load and improve concentration.

[0104] like Figure 3 As shown, the feedback optimization module is used to design intervention tasks based on comprehensive cognitive state data, adjust the task parameter data for the next stage, and achieve dynamic optimization of assessment tasks and cognitive level assessment through closed-loop feedback.

[0105] The feedback optimization module uses comprehensive cognitive state data (including cognitive load, attention trend, and task response score) combined with the caregiver's real-time performance to generate intervention task design data and adjust subsequent task parameters. Its implementation includes three main stages: intervention task design, task parameter adjustment, and closed-loop feedback optimization.

[0106] The feedback optimization module first analyzes the completion of the current task and the cognitive state of the caregiver based on the comprehensive cognitive state data. For example, a high cognitive load score (such as 90 points) may indicate that the current task is too complex, while a decreased attention trend score (such as 50 points) may reflect that the caregiver's attention is declining. The module generates intervention tasks based on these analysis results. The types of intervention tasks may include attention training tasks, memory load challenge tasks, or psychological relaxation tasks. Each task design has a clear goal. For example, an attention training task may be designed as a short visual search with a small number of targets and minimal background interference to reactivate the caregiver's attention.

[0107] The intervention task design generates task parameters through a specific task model, which include the number of goals, operation complexity, task time limit, and the degree of background distraction. For example, in a psychological relaxation task, the module may reduce the cognitive load of the caregiver by reducing the number of task goals, extending the task time limit, and removing visual or auditory distractors.

[0108] Based on the comprehensive cognitive state data and intervention task design results, the module dynamically adjusts the task parameters of the next stage to ensure that the difficulty, type, and interference characteristics of the subsequent tasks are compatible with the caregiver's current state. For example, in a task, if the caregiver's cognitive load score continues to increase but the task response score is stable, the module can adjust the task parameters to balance the challenge and completion rate, such as appropriately reducing the task difficulty (reducing the number of goals) or simplifying the operation steps.

[0109] The core of task parameter adjustment is to dynamically adjust the parameters using reward functions and adaptive optimization algorithms. The reward function guides the optimization strategy by evaluating the ratio of task completion rate and cognitive load, ensuring that the task design is challenging but does not overly increase cognitive burden. For example, optimizing the target recognition task by reducing the intensity of background interference can effectively alleviate the cognitive load of the caregiver without significantly reducing the task completion rate.

[0110] The feedback optimization module realizes the dynamic adjustment of assessment tasks and cognitive level assessment through a closed-loop feedback mechanism. The closed-loop feedback process includes collecting real-time data after task completion, updating the comprehensive cognitive state score, and feeding back the updated score to the task generation module. The module monitors key indicators during task execution in real time (such as task completion time, target accuracy, cognitive load signal), generates feedback data, and updates the comprehensive cognitive state score.

[0111] For example, in a visual memory task, if the closed-loop feedback shows that the caregiver's attention trend score has dropped significantly, the module will redesign the interference characteristics of the next stage of the task (such as reducing visual background interference) to improve attention concentration. At the same time, the closed-loop feedback mechanism ensures that the system can gradually optimize the task design strategy, thereby improving task adaptability and evaluation accuracy.

[0112] By adjusting task parameters in real time and designing personalized intervention tasks, the module can adapt to caregivers with different cognitive levels to improve task completion and engagement. For example, caregivers with low attention span can achieve higher completion rates by simplifying tasks or providing clear prompts.

[0113] The module effectively controls cognitive load by dynamically adjusting task parameters (such as the number of goals and interference characteristics) to prevent caregivers from becoming overly tired or frustrated due to tasks that are too difficult or too complex. For example, psychological relaxation tasks under high-load conditions can help relieve stress through deep breathing rhythm prompts or the introduction of relaxing audio.

[0114] Through the closed-loop feedback mechanism, the module can gradually optimize the task design strategy to ensure that subsequent tasks more accurately reflect the caregiver's cognitive level. For example, if the feedback shows that the task completion rate is low but the cognitive load is high, the system can optimize the time limit or prompt mechanism of the subsequent task to improve the accuracy of the assessment task.

[0115] Preferably, the feedback optimization module is used to generate intervention tasks based on the comprehensive cognitive state data, and the intervention tasks include attention training tasks, cognitive memory challenge tasks and psychological relaxation tasks; the attention training tasks improve the caregiver's concentration level by adjusting the number of visual targets and the target switching speed, the cognitive memory challenge tasks improve the caregiver's cognitive load tolerance by increasing the complexity of task interference elements, and the psychological relaxation tasks relieve the caregiver's decision-making pressure by controlling the deep breathing guidance rhythm and audio relaxation frequency.

[0116] The feedback optimization module uses comprehensive cognitive state data (including cognitive load, attention trend, and task response score) to analyze the current cognitive state of the caregiver, and dynamically generates intervention tasks that are highly matched to the caregiver's state by combining the target characteristics and parameter space of the intervention task. The core of the module is to map different intervention task types into specific parameter adjustment strategies, and provide precise intervention design through a dynamic task generation mechanism.

[0117] The attention training task aims to improve the caregiver's concentration level by adjusting the number of visual targets and the target switching speed. The principle is based on the dynamic adjustment of visual attention allocation. By setting the task parameters reasonably, the caregiver can maintain efficient attention during the rapid switching of visual targets. For example, increasing the number of targets can increase the complexity of the task, while shortening the target switching time can promote the training of rapid attention switching ability.

[0118] In specific implementation, the module collects the comprehensive cognitive status data of the caregiver. For example, a low attention trend score (such as 45 points) indicates distraction. Combined with the task completion rate and cognitive load level, the module generates the initial intervention task parameters: the number of targets is set to 3, and the switching speed is 2 seconds per target. During the execution of the task, the system monitors the changes in the caregiver's gaze point and the signal strength of the EEG Alpha frequency band in real time. If the concentration of attention gradually increases, the module will dynamically increase the number of targets (such as from 3 to 5) or shorten the switching speed (such as from 2 seconds to 1.5 seconds) to achieve dynamic adaptation of the task.

[0119] Cognitive memory challenge tasks are designed to improve caregivers' cognitive load tolerance by increasing the complexity of task interference elements. The design principle is to increase the difficulty of memory tasks through external interference (such as dynamic background, target ambiguity or sound interference), thereby enhancing caregivers' performance in high cognitive load environments.

[0120] When designing cognitive memory challenge tasks, the module dynamically adjusts interference elements based on the cognitive load score (such as a moderate 70 points) and task response score in the comprehensive cognitive state data. For example, when the cognitive load is moderate but the task response score is high (such as 80 points), the module can increase the complexity of interference factors, such as adding dynamic visual backgrounds, increasing the number of targets, or adding random interference patterns between targets. Interference adjustment is based on the caregiver's real-time EEG Beta frequency band signal and task completion rate to ensure that the intervention can appropriately increase cognitive load while maintaining the completeability of the task.

[0121] The psychological relaxation task aims to relieve the decision-making pressure of the caregiver and restore his / her cognitive state by controlling the rhythm of deep breathing guidance and the frequency of audio relaxation. The principle is based on the theory of autonomic nervous system regulation. By guiding the caregiver to take regular deep breaths and providing soothing audio, the sympathetic nerve activity is reduced and the parasympathetic nerve function is enhanced, thereby effectively relieving cognitive load and psychological pressure.

[0122] The module generates the initial parameters of the psychological relaxation task based on the cognitive load score (such as above 85 points) and the attention trend score (such as a significant drop to 40 points) in the comprehensive cognitive state data. For example, the deep breathing guidance rhythm is set to 6 times per minute, and the audio relaxation frequency is set to a soft frequency band of 60Hz to 120Hz. During the execution of the task, the system dynamically adjusts the deep breathing guidance rhythm (such as from 6 times per minute to 5 times per minute) or switches the audio frequency range by collecting the caregiver's heart rate changes and pupil diameter fluctuations in real time to further enhance the intervention effect.

[0123] The attention training task enabled caregivers to gradually enhance visual attention and improve task completion by adjusting the number of targets and switching speed. For example, after completing a target search task in a highly dynamic background, the attention trend score increased from 45 to 70.

[0124] The cognitive memory challenge task improves the caregivers’ cognitive flexibility and anti-interference ability through complex interference design, gradually improving their performance in high cognitive load tasks. For example, by adding dynamic background and random interference patterns, the caregivers’ task response scores gradually increased from 70 to 85.

[0125] The psychological relaxation task significantly reduced the cognitive load and decision-making pressure of caregivers through physiological signal guidance and audio intervention. For example, in the deep breathing and relaxation audio task, the cognitive load score dropped from 85 to 65, while the attention trend score increased from 40 to 60.

[0126] Taking the attention training task performed by the caregiver of a cerebral infarction patient as an example, the initial comprehensive cognitive state data showed that the attention trend score was 40 points and the cognitive load score was 75 points. The module generated the initial task parameters of 3 targets and 2 seconds of switching speed. During the execution of the task, the real-time collected eye movement data showed that the concentration of the gaze point increased, the EEG Alpha frequency band signal was enhanced, and the attention trend score rose to 55 points. The module dynamically adjusted the task parameters, increased the number of targets to 5, and shortened the switching speed to 1.5 seconds. Through multiple rounds of tasks, the caregiver's attention trend score further increased to 70 points, and the task completion rate was significantly improved.

[0127] Preferably, the feedback optimization module further includes an intervention effect analysis unit, and the data collected by the intervention effect analysis unit includes intervention completion rate data, real-time physiological response data and behavior adjustment data; the comprehensive cognitive state data before and after the intervention are compared and analyzed through a multimodal regression analysis model to generate intervention effect evaluation data, and the intervention effect evaluation data is used to optimize the task parameter data for the next stage.

[0128] The intervention effect analysis unit uses the intervention completion rate, real-time physiological response data, and behavioral adjustment data as inputs, and compares and analyzes these data with the comprehensive cognitive state data before and after the intervention through a multimodal regression analysis model to generate evaluation data reflecting the effectiveness of the intervention task. Its core principle is to use a regression model to quantify the contribution of each input data to the change in the comprehensive cognitive state, so as to derive the actual effect of the intervention task in improving attention, alleviating cognitive load, or optimizing task response scores.

[0129] The intervention completion rate data collected by the intervention effect analysis unit reflects the caregiver's adaptability and participation in the intervention task, such as the proportion of the number of visual targets correctly completed in the attention training task to the total number of targets. Real-time physiological response data reflects the caregiver's physiological state through EEG signals (such as Beta frequency band changes), heart rate variability (HRV), and pupil diameter dynamic changes. Behavioral adjustment data includes behavioral trajectory stability, reaction time, and mouse click deviation, indicating changes in behavioral performance during the task.

[0130] The collected data are first standardized to eliminate the scale differences between different data sources. For example, the frequency band power changes of the EEG signal are normalized to the range of 0-1, and the mouse trajectory deviation is quantified as the average deviation distance per unit time. The standardized data is aligned with the comprehensive cognitive state data through a time synchronization algorithm to ensure the consistency of multimodal input.

[0131] The intervention effect analysis unit uses a multimodal regression analysis model to compare the comprehensive cognitive state data before and after the intervention. The input of the model is the intervention completion rate, real-time physiological response data and behavioral adjustment data, and the output is the intervention effect evaluation data, including the quantitative results of the improvement of attention concentration level, cognitive load relief degree and task response performance.

[0132] The multimodal regression analysis model adopts a weighted regression strategy to dynamically assign weights according to the characteristics and reliability of each input data. For example, in a psychological relaxation task, heart rate variability and EEG signals have higher weights because these physiological data can directly reflect the degree of relief of psychological stress; in a cognitive memory challenge task, behavioral adjustment data (such as reaction time) may have a higher weight because it has a more significant impact on the quality of task completion.

[0133] The model calculates the intervention effect score by analyzing the trend of comprehensive cognitive state data before and after the intervention task. For example, the prognostic attention trend score increased from 45 to 70 points, and the cognitive load score decreased from 85 to 65 points. The model will comprehensively conclude that the intervention effect score is 80 points (out of 100 points), indicating that the intervention task is effective in improving attention concentration and alleviating cognitive load.

[0134] Based on the intervention effect evaluation data, the intervention effect analysis unit provides input to the feedback optimization module for dynamically adjusting the task parameter data for the next stage. For example, several pre-effect evaluations show that the best effect is achieved when the target number of attention training tasks is set to 3, but the switching speed is too slow (such as 2 seconds). The module will optimize the next task parameters, keep the target number at 3, and adjust the switching speed to 1.5 seconds to further improve attention concentration.

[0135] Through regression analysis of multimodal data, the unit was able to accurately quantify the effect of the intervention task on the cognitive state of the caregiver. For example, after the attention training task, the improvement in the attention trend score clearly reflected the effect of the task.

[0136] By inputting intervention effect evaluation data into the feedback optimization module, the unit can dynamically adjust the task design so that subsequent tasks can more accurately adapt to the caregiver's cognitive state.

[0137] The unit improves the system's adaptability in different intervention scenarios through dynamic analysis of real-time physiological response data and behavioral adjustment data, while reducing the instability caused by improper parameter settings in task design.

[0138] Taking the cognitive memory challenge task performed by the caregiver of a cerebral infarction patient as an example, the data collected by the intervention effect analysis unit include the intervention completion rate (correct target recognition rate is 70%), real-time physiological response data (significant enhancement of EEG Beta band power and reduction of heart rate variability) and behavioral adjustment data (the average reaction time is shortened to 1.8 seconds). The multimodal regression analysis model calculates the correlation between these data and the comprehensive cognitive state data (the cognitive load score before and after the intervention dropped from 80 points to 65 points, and the task response score increased from 60 points to 75 points), and the intervention effect score is 85 points. Based on this evaluation, the module adjusts the task parameters for the next stage, increases the number of targets to 5, and moderately reduces the complexity of interference elements to further optimize the task design.

[0139] Preferably, the feedback optimization module transmits the intervention effect evaluation data and the adjusted next stage task parameter data to the task generation module through a closed-loop feedback mechanism, which is used to dynamically adjust the task type, task difficulty and interference characteristics of subsequent evaluation tasks, thereby achieving dynamic adaptation of the evaluation tasks to the caregiver's cognitive state.

[0140] The core of the closed-loop feedback mechanism is to transmit the intervention effect evaluation data (such as task completion rate, cognitive load relief, improvement in concentration level, etc.) and the adjusted task parameter data (including task type, task difficulty and interference characteristics) to the task generation module in real time, so that it can dynamically adjust the subsequent task design. The feedback optimization module performs the following key steps in this process:

[0141] The feedback optimization module first generates intervention effect evaluation data through the intervention effect analysis unit, including the degree of improvement in concentration of attention training tasks, the degree of improvement in load tolerance of cognitive memory challenge tasks, and the stress relief effect of psychological relaxation tasks. Subsequently, the module dynamically adjusts the task parameter data according to the evaluation results. For example, when the concentration score increases from 45 to 70 and the task completion rate reaches 80%, the module maintains the task difficulty at the current level, appropriately increases the number of targets (such as from 5 to 7), and reduces interference characteristics (such as reducing background dynamics).

[0142] The adjusted task parameter data is transmitted to the task generation module through a closed-loop feedback mechanism. The task generation module combines it with the current multimodal feature data to dynamically generate the type and parameters of the subsequent task. For example, a visual recognition task is generated based on the caregiver's attention concentration tendency. The number of targets and the switching speed are optimized to 7 targets and 1.5 seconds per target according to the adjusted parameters.

[0143] The task generation module uses closed-loop feedback data to update the task design in real time to ensure that subsequent tasks can accurately adapt to the cognitive state of the caregiver. For example, when the intervention effect evaluation shows that the caregiver's performance is significantly reduced in a high-interference environment (such as the cognitive load score rises to 85), the task generation module will prioritize reducing interference characteristics while optimizing task difficulty to make the task challenging while maintaining completion rate.

[0144] By closely linking the intervention effect evaluation data with the task generation module, the system can accurately adjust the task type, difficulty and interference characteristics according to the real-time status of the caregiver. For example, when the attention training task shows significant results, the system can gradually increase the number of targets and switching speed to further improve the level of concentration.

[0145] The closed-loop feedback mechanism ensures that the effects of the intervention task can be continued and strengthened in the subsequent evaluation task. For example, by providing feedback on the optimized psychological relaxation task, the system can gradually reduce the cognitive load of the caregiver and provide a better starting point for the subsequent task.

[0146] The closed-loop feedback mechanism realizes the real-time linkage between the task generation module, the intervention effect analysis unit and the feedback optimization module, so that the evaluation process can adapt to the dynamic changes of the caregiver's state. For example, in the case of a decline in the concentration trend, the system adjusts the task type in time through the feedback mechanism, giving priority to generating low-load attention training tasks to ensure the continuity of the evaluation.

[0147] Taking the dynamic assessment of a caregiver of a patient with cerebral infarction as an example, the initial task is a cognitive memory challenge task, and the task parameters include the number of targets of 5 and medium dynamic background interference. The intervention effect analysis unit shows that the task completion rate is 60%, the cognitive load score is 85, and the attention concentration score is 50. Based on this result, the feedback optimization module optimizes the task parameters, adjusts the number of targets to 4, reduces the intensity of background interference to static, and switches the task type to an attention training task. Through closed-loop feedback, the task generation module receives the optimization parameters and generates the target number of 4 for subsequent tasks, the switching speed is 1.5 seconds, and the static background interference. After the subsequent tasks are completed, the system optimizes the task parameters again through feedback to form a continuous dynamic evaluation process.

[0148] like Figure 4 As shown, a method for executing the cognitive level assessment system applied to caregivers of cerebral infarction patients comprises the following steps:

[0149] Collect EEG data, behavioral data, speech data and eye movement data through a multimodal data acquisition module; perform frequency band decomposition on the EEG data to extract Delta band, Theta band, Alpha band and Beta band signals; perform time synchronization and outlier removal on the behavioral data; perform filtering and denoising on the speech data to extract intonation, speech speed and pause features; perform Gaussian smoothing on the eye movement data to extract pupil diameter changes and gaze point position information; align the above processed data through a time synchronization algorithm to generate multimodal feature data;

[0150] Based on the multimodal feature data, task parameter data is generated through a task generation module, and the task parameter data includes task type, task difficulty and interference characteristics; the task parameters are generated by using a generator in a conditional generative adversarial network, and the generated task parameters are verified and optimized by a discriminator; according to the task execution status, the task parameter data is optimized in real time through a reinforcement learning optimization module, and a reward function is used to balance and optimize the task completion rate and cognitive load level, and the reward function is:

[0151]

[0152] Among them, R represents the reward value, C represents the task completion rate, and L represents the cognitive load level;

[0153] Based on the multimodal feature data and task parameter data, a comprehensive cognitive state is extracted through a dual-stream deep learning network in a cognitive state decoding module, wherein the comprehensive cognitive state includes cognitive load, attention trend and task response score; the first data stream is used to process the time series features of EEG data and behavioral data, and the second data stream is used to process the multimodal features of speech data and eye movement data; the decoding results of the first data stream and the second data stream are used to generate comprehensive cognitive state data through a feature fusion model; a dynamic Bayesian network is used to perform state transition modeling based on the comprehensive cognitive state data to predict future cognitive ability change trends, and the state transition modeling formula is:

[0154]

[0155] Among them, P(S t |S t-1 ) indicates the current state S t In the previous state S t-1 The transition probability under X i represents multimodal input features; P(X i ) represents feature X i The probability distribution of P(S t |S t-1 ,X i ) represents the current state to the previous state and feature X i The conditional probability of

[0156] Intervention tasks are designed based on comprehensive cognitive state data, and the intervention tasks include attention training tasks, cognitive memory challenge tasks, and psychological relaxation tasks; intervention task parameters are adjusted, including the number of visual targets, the complexity of task interference elements, and the rhythm of deep breathing guidance; feedback data after the completion of the intervention task is collected, including intervention completion rate data, physiological response data, and behavior adjustment data, and intervention effect evaluation data is generated using a multimodal regression analysis model; based on the intervention effect evaluation data and the adjusted task parameter data, the task parameters of the next stage are optimized through a closed-loop feedback mechanism, and the task type, task difficulty, and interference characteristics of subsequent tasks are dynamically adjusted to achieve dynamic adaptation of the evaluation task to the cognitive state of the caregiver.

[0157] The present invention acquires EEG, behavior, speech and eye movement data through a multimodal data acquisition module, performs synchronous preprocessing on the data, extracts feature information to generate multimodal feature data. These feature data dynamically generate task parameters through a task generation module, including task type, difficulty and interference characteristics, and optimize task parameters through a conditional generative adversarial network. At the same time, the task execution is optimized in real time in combination with a reinforcement learning module, and a reward function is used to balance the task completion rate and cognitive load.

[0158] The cognitive state decoding module processes multimodal feature data and task parameter data through a dual-stream deep learning network to generate a comprehensive cognitive state (cognitive load, attention trend, task response score). This module uses the first data stream to decode the time series features of EEG and behavioral data, and the second data stream to decode the multimodal features of speech and eye movement data, and integrates the decoding results through a feature fusion model, and further uses a dynamic Bayesian network to model state transitions and predict the trend of cognitive ability changes.

[0159] Based on comprehensive cognitive state data, the system designs personalized intervention tasks, including attention training, cognitive memory challenges, and psychological relaxation tasks, and optimizes the intervention effect by adjusting the number of visual targets, interference complexity, and deep breathing rhythm. By collecting intervention completion rate, physiological response, and behavioral adjustment data, multimodal regression analysis is used to generate intervention effect evaluation data, and a closed-loop feedback mechanism is combined to optimize the next stage of task design to achieve dynamic adaptation of task type, difficulty, and interference characteristics.

[0160] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A cognitive level assessment system for caregivers of patients with cerebral infarction, characterized in that: include: A multimodal data acquisition module is used to collect EEG data, behavioral data, speech data, and eye movement data, and synchronize and preprocess the collected data to generate multimodal feature data; A task generation module, used to generate task parameter data based on the multimodal feature data, wherein the task parameter data includes task type, task difficulty and interference characteristics, and optimize the task parameter data according to the cognitive state of the caregiver to start the assessment task; A cognitive state decoding module, used to decode the comprehensive cognitive state of the caregiver based on the multimodal feature data and the task parameter data, and generate comprehensive cognitive state data, wherein the comprehensive cognitive state data includes cognitive load, attention trend and task response score; The feedback optimization module is used to design intervention tasks based on comprehensive cognitive state data, adjust the task parameter data for the next stage, and achieve dynamic optimization of assessment tasks and cognitive level assessment through closed-loop feedback.

2. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 1, characterized in that: The multimodal data acquisition module includes an EEG signal acquisition device, a behavior capture device, a voice recording device and an eye tracking device, which are used to respectively collect EEG data, behavior data, voice data and eye movement data; the signals recorded by the EEG signal acquisition device include Delta frequency band signals, Theta frequency band signals, Alpha frequency band signals and Beta frequency band signals; the data recorded by the behavior capture device include mouse trajectory, click position and reaction time; the data collected by the voice recording device include intonation, speech speed and pauses; the data recorded by the eye tracking device include pupil diameter changes and gaze point positions.

3. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 1, characterized in that: The task generation module includes a conditional generative adversarial network, which includes a generator and a discriminator. The generator generates task parameter data based on multimodal feature data, and the task parameter data includes task type, task difficulty and interference characteristics; the discriminator is used to verify the generated task parameter data and optimize the generated task parameter data by adjusting the parameters of the generator.

4. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 1, characterized in that: The task generation module further includes a reinforcement learning optimization module, which optimizes the task parameter data based on the task completion status. The optimization is achieved through a reward function, and the standard form of the reward function is: Among them, R represents the reward value, C represents the task completion rate, and L represents the cognitive load level.

5. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 1, characterized in that: The cognitive state decoding module includes a dual-stream deep learning network, which is used to decode the comprehensive cognitive state based on multimodal feature data and task parameter data, and the comprehensive cognitive state includes cognitive load, attention trend and task response score; the dual-stream deep learning network includes a first data stream and a second data stream, the first data stream processes the time series features of EEG data and behavioral data through a self-attention mechanism, and the second data stream processes the multimodal features of speech data and eye movement data through a convolutional neural network, and the decoding results of the first data stream and the second data stream are used to generate comprehensive cognitive state data through a feature fusion model.

6. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 1, characterized in that: The cognitive state decoding module further includes a dynamic Bayesian network, which is used to predict the trend of cognitive ability changes based on the comprehensive cognitive state data. The state transition formula of the dynamic Bayesian network is: Among them, P(S t |S t-1 ) indicates the current state S t In the previous state S t-1 The transition probability under X i represents multimodal input features; P(X i ) represents feature X i The probability distribution of P(S t |S t-1 , X i ) represents the current state to the previous state and feature X i The conditional probability of .

7. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 1, characterized in that: The feedback optimization module is used to generate intervention tasks based on the comprehensive cognitive state data, wherein the intervention tasks include attention training tasks, cognitive memory challenge tasks and psychological relaxation tasks; The attention training task improves the caregivers' concentration level by adjusting the number of visual targets and the target switching speed. The cognitive memory challenge task improves the caregivers' cognitive load tolerance by increasing the complexity of task interference elements. The psychological relaxation task relieves the caregivers' decision-making pressure by controlling the deep breathing guidance rhythm and audio relaxation frequency.

8. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 7, characterized in that: The feedback optimization module further includes an intervention effect analysis unit, which collects data including intervention completion rate data, real-time physiological response data and behavior adjustment data; a multimodal regression analysis model is used to compare and analyze the comprehensive cognitive state data before and after the intervention to generate intervention effect evaluation data, and the intervention effect evaluation data is used to optimize the task parameter data for the next stage.

9. The cognitive level assessment system for caregivers of cerebral infarction patients according to claim 8, characterized in that: The feedback optimization module transmits the intervention effect evaluation data and the adjusted next-stage task parameter data to the task generation module through a closed-loop feedback mechanism, which is used to dynamically adjust the task type, task difficulty and interference characteristics of subsequent evaluation tasks, thereby achieving dynamic adaptation of the evaluation tasks to the caregiver's cognitive state.

10. A method for executing the cognitive level assessment system for caregivers of cerebral infarction patients according to any one of claims 1 to 9, characterized in that: The following steps are involved: Collect EEG data, behavioral data, speech data and eye movement data through a multimodal data acquisition module; perform frequency band decomposition on the EEG data to extract Delta band, Theta band, Alpha band and Beta band signals; perform time synchronization and outlier removal on the behavioral data; perform filtering and denoising on the speech data to extract intonation, speech speed and pause features; perform Gaussian smoothing on the eye movement data to extract pupil diameter changes and gaze point position information; align the above processed data through a time synchronization algorithm to generate multimodal feature data; Based on the multimodal feature data, task parameter data is generated through a task generation module, and the task parameter data includes task type, task difficulty and interference characteristics; the task parameters are generated by using a generator in a conditional generative adversarial network, and the generated task parameters are verified and optimized by a discriminator; according to the task execution status, the task parameter data is optimized in real time through a reinforcement learning optimization module, and a reward function is used to balance and optimize the task completion rate and cognitive load level, and the reward function is: Among them, R represents the reward value, C represents the task completion rate, and L represents the cognitive load level; Based on the multimodal feature data and task parameter data, a comprehensive cognitive state is extracted through a dual-stream deep learning network in a cognitive state decoding module, wherein the comprehensive cognitive state includes cognitive load, attention trend and task response score; the first data stream is used to process the time series features of EEG data and behavioral data, and the second data stream is used to process the multimodal features of speech data and eye movement data; the decoding results of the first data stream and the second data stream are used to generate comprehensive cognitive state data through a feature fusion model; a dynamic Bayesian network is used to perform state transition modeling based on the comprehensive cognitive state data to predict future cognitive ability change trends, and the state transition modeling formula is: Among them, P(S t |S t-1 ) indicates the current state S t In the previous state S t-1 The transition probability under X i represents multimodal input features; P(X i ) represents feature X i The probability distribution of P(S t |S t-1 , X i ) represents the current state to the previous state and feature X i The conditional probability of Intervention tasks are designed based on comprehensive cognitive state data, and the intervention tasks include attention training tasks, cognitive memory challenge tasks, and psychological relaxation tasks; intervention task parameters are adjusted, including the number of visual targets, the complexity of task interference elements, and the rhythm of deep breathing guidance; feedback data after the completion of the intervention task is collected, including intervention completion rate data, physiological response data, and behavior adjustment data, and intervention effect evaluation data is generated using a multimodal regression analysis model; based on the intervention effect evaluation data and the adjusted task parameter data, the task parameters of the next stage are optimized through a closed-loop feedback mechanism, and the task type, task difficulty, and interference characteristics of subsequent tasks are dynamically adjusted to achieve dynamic adaptation of the evaluation task to the cognitive state of the caregiver.

Images