Cognitive competence assessment method, training method, system, equipment and medium

Abstract

The invention belongs to the technical field of cognitive power, and relates to a cognitive competence assessment method, training method, system, equipment and medium, the system comprises a data acquisition unit used for acquiring electrophysiological signal data and behavior data of a user; the evaluation unit is connected with the data acquisition unit and is used for executing a cognitive evaluation scale task and synchronously acquiring the electrophysiological signal data, the behavior data and the scale score of the user; the fusion analysis unit is connected with the data acquisition unit and the evaluation unit and is used for integrating the cognitive evaluation scale data, the electrophysiological signals and the behavior data and inputting the data into a multi-modal fusion evaluation model so as to output a cognitive load index and an emotional stability score; and the training unit is connected with the fusion analysis unit and is used for adjusting the VR training difficulty according to the cognitive load index and the emotion stability score. According to the scheme of the invention, the accuracy of cognitive ability evaluation and the applicability of cognitive training are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cognitive power. More particularly, the present application relates to a cognitive ability evaluation method, a training method, a system, a device and a medium. BACKGROUND

[0002] Cognitive ability refers to the ability of the human brain to process, store and extract information, that is, the ability of people to grasp the composition, performance and relationship with other things, development power, development direction and basic laws of things, such as understanding, attention, thinking and imagination. The ability is considered to be cognitive ability.

[0003] The learning behavior, ability level and understanding degree exhibited by the learner in the learning process are closely related to the cognitive ability of the learner, and according to the cognitive ability of the learner, the learner can be more targeted and more efficient to know the learning or thinking of the learner. The current traditional cognitive evaluation methods include neuropsychological test set, MoCA Montreal cognitive function evaluation scale and other methods, and nuclear magnetic technology and electroencephalogram technology are also used to measure the corresponding characteristic data for research.

[0004] The limitations of traditional cognitive evaluation include dependence on subjective scale (such as MMSE and MoCA), susceptibility to subject state and evaluator bias, inability to monitor cognitive load and emotional state (such as anxiety and fatigue) in real time, and deficiencies in game training system. And the training difficulty in the traditional cognitive evaluation method is fixed, lacking a mechanism to dynamically adapt to the physiological state of the user, and the training effect is only evaluated by behavior data (such as reaction time and accuracy), ignoring the changes in the neurophysiological level.

[0005] Therefore, how to solve the problem of inaccurate cognitive ability evaluation is one of the key research directions. SUMMARY

[0006] To solve the above technical problem of inaccurate cognitive ability evaluation, the present application provides a solution in the following aspects.

[0007] In a first aspect, the present application provides a VR-based cognitive ability evaluation and training system, comprising: a data acquisition unit for acquiring electro-physiological signal data and behavior data of a user; an evaluation unit connected with the data acquisition unit, for executing a cognitive evaluation scale task and synchronously acquiring electro-physiological signal data, behavior data and scale scores of the user; a fusion analysis unit connected with the data acquisition unit and the evaluation unit, for integrating cognitive evaluation scale data, electro-physiological signal and behavior data and inputting a multi-modal fusion evaluation model to output a cognitive load index and an emotional stability score; and a training unit connected with the fusion analysis unit, for adjusting VR training difficulty according to the cognitive load index and the emotional stability score.

[0008] In one embodiment, the electrophysiological signal data comprises EEG signals and ECG signals, and the behavioral data comprises reaction time.

[0009] In one embodiment, the evaluation unit is further configured to: band-pass filter and denoise the electrophysiological signals; and perform power spectral density calculation and frequency band division on the denoised electrophysiological signals to calculate the θ / α power ratio and the ratio of low frequency band power to high frequency band power.

[0010] In a second aspect, the present application also provides a cognitive ability evaluation method, applied to the VR-based cognitive ability evaluation and training system of one or more embodiments of the first aspect, the method comprising: presenting cognitive scale tasks in a VR environment; performing cognitive evaluation scale tasks and synchronously acquiring electrophysiological signal data and behavioral data of a user; integrating the cognitive evaluation scale data, the electrophysiological signals and the behavioral data and inputting them into a trained multi-modal fusion evaluation model to output a cognitive load index and an emotional stability score.

[0011] In one embodiment, the integration of the cognitive evaluation scale data, the electrophysiological signals and the behavioral data comprises: extracting scores of the cognitive evaluation scale tasks; and pre-processing and frequency band dividing the electrophysiological signals to extract EEG signal features and ECG signal features, wherein the EEG signal features comprise a θ / α power ratio and the ECG signal features comprise HRV frequency domain indicators.

[0012] In one embodiment, the pre-processing and frequency band dividing of the electrophysiological signals to extract the EEG signal features and the ECG signal features comprises: band-pass filtering and denoising the electrophysiological signals; and performing power spectral density calculation and frequency band division on the denoised electrophysiological signals to calculate the θ / α power ratio and the HRV frequency domain indicators.

[0013] In a third aspect, the present application also provides a cognitive training method, applied to the VR-based cognitive ability evaluation and training system of one or more embodiments of the first aspect, the method comprising: acquiring a cognitive load index and an emotional stability score; generating a corresponding training scheme based on the cognitive load index and the emotional stability score; acquiring a θ / α power ratio and a ratio of low frequency band power to high frequency band power in real time, and performing environment simplification and / or decompression intervention in combination with the θ / α power ratio and the ratio of low frequency band power to high frequency band power.

[0014] In one embodiment, the environment simplification includes three-level environment simplification: in response to the theta / alpha power ratio being in [1.5, 2) or LF / HF being in [3, 4), primary simplification is performed, VR particle special effects are turned off, and target highlighting is enhanced; LF / HF is the ratio of low-frequency band power to high-frequency band power; in response to the theta / alpha power ratio being in [2, 2.5) or LF / HF being in [4, 5), intermediate simplification is performed, the 3D scene is reduced to 2D, and the number of task targets is halved; and in response to the theta / alpha power ratio being in [2.5, ∞) or LF / HF being in [5, ∞), complete simplification is performed, a static background is switched to, and automatic aiming and breathing guidance animations are enabled.

[0015] In a fourth aspect, the present application also provides an electronic device, comprising: a processor; and a memory storing executable instructions of the processor, which, when executed by the processor, implement a cognitive ability evaluation method as described above, or implement a cognitive training method as described above.

[0016] In a fifth aspect, the present application also provides a computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implement a cognitive ability evaluation method as described above, or implement a cognitive training method as described above.

[0017] The present application has the beneficial effect that, according to the scheme of the present application, the training result analysis of the bioelectricity acquisition module assisted evaluation scale result analysis and game training is performed, so that the dynamic data evaluation is combined on the basis of the static scale analysis, the neurophysiological level change is combined with the static evaluation method, and the accuracy of the cognitive ability evaluation and the applicability of the cognitive training are effectively improved.

[0018] Further, the present application also realizes the accurate evaluation of the cognitive ability by using the power characteristics of the corresponding frequency band in the acquired electrophysiological signal data to determine the cognitive state.

[0019] Still further, the present application also realizes the applicability of the VR training to different cognitive states through the three-level environment simplification process, and improves the training effect. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which a number of embodiments of the application are illustrated by way of example and not limitation. Like or corresponding elements in the drawings are denoted by like reference numerals, and:

[0021] Figure 1 is a schematic diagram showing an exemplary structure of an integrated machine for realizing cognitive ability evaluation and training according to an embodiment of the present application;

[0022] Figure 2 is a schematic diagram showing the functional principle of an all-in-one machine for implementing cognitive ability assessment and training according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram showing the composition of a cognitive ability assessment and training system according to the present application;

[0024] Figure 4 is a flowchart showing a cognitive ability assessment method according to the present application;

[0025] Figure 5 is a flowchart showing a cognitive training method according to the present application;

[0026] Figure 6 is a schematic diagram showing the composition structure of an electronic device according to the present application;

[0027] Figure 1 In the figure, 1 is an all-in-one machine computer, 2 is a trolley, 3 is a printer, and 4 is a VR headset. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0029] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 is a schematic diagram showing an exemplary structure of an all-in-one machine for implementing cognitive ability assessment and training according to an embodiment of the present application. Figure 2 is a schematic diagram showing the functional principle of an all-in-one machine for implementing cognitive ability assessment and training according to an embodiment of the present application.

[0031] As shown in Figure 1 , the all-in-one machine for implementing cognitive ability assessment and training in the present application mainly comprises an all-in-one machine computer 1, a VR headset 4 (integrating EEG dry electrodes, collecting from the frontal lobe / temporal region), a bioelectricity acquisition module (EEG / ECG), a data processing terminal (VR cognitive ability assessment and training system), a handle, a printer 3, and a trolley 2. The cognitive VR ability assessment and training system installed on the all-in-one machine is used to perform assessment and issue training and view user records. The bioelectricity acquisition module (EEG / ECG) acquires the electrophysiological signal data of the user, realizes auxiliary analysis of the assessment results and game training results, and can record user record data.

[0032] Specifically, in the system, the bioelectricity collection module includes a wearable ECG module (wristband type, supporting HRV analysis) and an EEG (Electroencephalogram) module to collect bioelectricity physiological signals (referred to as electrophysiological signals in the present application). The VR head-mounted device integrates an EEG dry electrode and collects brain wave signals through the frontal lobe / temporal region. After the collection of the electrophysiological signal data is completed, the evaluation unit and the training unit in the cognitive ability evaluation and training system are used to realize the evaluation and training of the cognitive ability of the user.

[0033] As shown in Figure 2 , the VR head-mounted device integrates an OLED display screen (double-eye 4K), a 6DoF tracking camera, and an audio output module. The bioelectricity collection module has an EEG dry electrode array, which is arranged on the VR head-mounted device, mainly in the frontal lobe and the temporal region. The bioelectricity collection module also includes a wearable ECG module, such as a wristband type ECG sensor. At the same time, the bioelectricity collection module is also provided with a signal conditioning circuit, which can process the audio output by the VR head-mounted device. The data processing terminal can include a cloud server (AI analysis), an edge computing device (FPGA / GPU), etc., to realize the generation of the cognitive ability evaluation and training scheme.

[0034] The core of the present application is the implementation of cognitive ability evaluation and training. Next, the specific composition and functions of the data processing terminal (i.e., the cognitive ability evaluation and training system based on VR) will be described in detail. Figure 3 is a schematic diagram showing the composition of the cognitive ability evaluation and training system according to the present application.

[0035] As shown in Figure 3 , the cognitive ability evaluation and training system based on VR includes a data collection unit for collecting electrophysiological signal data and behavior data of the user. In some embodiments, the data collection unit can be connected with various devices in the aforementioned bioelectricity collection module, so as to obtain the bioelectricity signal data and behavior data of the user, etc. In some embodiments, the electrophysiological signal data includes EEG signals and ECG signals, and the behavior data includes reaction time.

[0036] An evaluation unit connected with the data collection unit is configured to perform cognitive evaluation scale tasks and synchronously obtain the electrophysiological signal data, behavior data, and scale scores of the user. In some embodiments, the evaluation unit is also configured to perform band-pass filtering and denoising on the electrophysiological signals. The denoised electrophysiological signals are subjected to power spectral density calculation and frequency band division to calculate the theta / alpha power ratio and the ratio of low frequency band power to high frequency band power.

[0037] a fusion analysis unit connected with the data acquisition unit and the evaluation unit, for integrating the cognitive evaluation scale data, the electrophysiological signals and the behavior data and inputting a multi-modal fusion evaluation model to output a cognitive load index and an emotional stability score.

[0038] a training unit connected with the fusion analysis unit, for adjusting the VR training difficulty according to the cognitive load index and the emotional stability score.

[0039] The working principle of the above system is as follows: first, EEG signals and ECG signals are collected, then signal preprocessing is performed, including filtering, noise reduction, etc., then signal feature extraction is performed, such as θ / α wave, HRV, etc., after feature extraction, cognitive ability evaluation is performed by using a multi-modal fusion evaluation model, and a corresponding training mode is selected according to the evaluation result. The training mode may be, for example, real-time adjustment of VR task parameters. In addition, a comprehensive evaluation report can also be directly generated according to the evaluation result, so as to facilitate the user to understand the cognitive state.

[0040] Next, the above evaluation process and training process will be introduced in detail.

[0041] Figure 4 is a flowchart showing a cognitive ability evaluation method according to the present application.

[0042] As shown in Figure 4 , at step S401, a cognitive scale task is presented in a VR environment. In some embodiments, the cognitive scale can adopt a cognitive traditional evaluation scale. The user can complete a preset task (such as remembering the position of a card, a quick reaction test, etc.) according to the items in the cognitive scale task, and score the scale in combination with classic index scoring, etc., for example, the MoCA total score and sub-item score can be calculated, etc., to evaluate the overall cognitive function.

[0043] At step S402, the cognitive evaluation scale task is executed and the electrophysiological signal data and behavior data of the user are synchronously acquired. In some embodiments, the user completes the cognitive traditional evaluation scale, and the EEG (θ / α power ratio, P300 latency), ECG (HRV) and behavior data (reaction time) are synchronously recorded by using dry electrodes (such as frontal lobe Fp1 / Fp2 leads) and wristband optical sensors (PPG), and the hardware trigger (TTL pulse) is aligned with the scale task start / end time stamp. By integrating the scale data, the electrophysiological signals (EEG / ECG), the behavior data,

[0044] At step S403, the cognitive assessment scale data, the electrophysiological signals and the behavioral data are integrated and input into the trained multi-modal fusion assessment model to output the cognitive load index and the emotional stability score. In some embodiments, the electrophysiological signals include EEG signals and ECG signals, and the behavioral data includes reaction time. The integration of the cognitive assessment scale data, the electrophysiological signals and the behavioral data includes: extracting scores of cognitive assessment scale tasks. The electrophysiological signals are preprocessed and frequency band divided to extract EEG signal features and ECG signal features, wherein the EEG signal features include a theta / alpha power ratio, and the ECG signal features include HRV frequency domain indicators.

[0045] Specifically, the process of extracting the EEG signal features and the ECG signal features includes band-pass filtering and denoising the electrophysiological signals. The denoised electrophysiological signals are subjected to power spectral density calculation and frequency band division to calculate the theta / alpha power ratio and the HRV frequency domain indicators.

[0046] The process of extracting the theta / alpha power ratio or the P300 latency feature includes:

[0047] I. Signal preprocessing:

[0048] (1) Filtering: Band-pass filtering (theta wave: 4-7 Hz; alpha wave: 8-13 Hz), removing power frequency interference (50 / 60 Hz) and electromyographic artifacts.

[0049] (2) Denoising: Independent component analysis (ICA) or wavelet transform to remove electrooculogram (EOG) and electrocardiogram (ECG) artifacts.

[0050] (3) Segmentation: Dividing EEG segments according to task stages (e.g., resting state, VR task).

[0051] II. Power spectral density (PSD) calculation:

[0052] (1) Calculating power spectrum using fast Fourier transform (FFT) or Welch method.

[0053] (2) Extracting average power values of theta band (4-7 Hz) and alpha band (8-13 Hz).

[0054] (3) Theta / alpha power ratio calculation: Theta / alpha power ratio = average theta power / average alpha power. It can be calculated for specific electrodes (e.g., Pz) or whole brain multi-electrode mean.

[0055] Extraction of HRV frequency domain indicator (LF / HF ratio) feature:

[0056] I. Frequency band division:

[0057] (1) Very low frequency (VLF): 0.003-0.04Hz (usually related to factors such as body temperature regulation, with limited clinical significance).

[0058] (2) Low frequency (LF): 0.04-0.15Hz (reflects mixed activity of sympathetic and parasympathetic nerves, which needs to be normalized).

[0059] (3) High frequency (HF): 0.15-0.4Hz (reflects parasympathetic nerve activity and is related to respiratory rhythm).

[0060] II. Calculate the power of the frequency band:

[0061] Integrate the power spectral areas within the LF and HF bands to obtain LF_power and HF_power.

[0062] (1) Normalization (eliminating individual differences):

[0063] LF_nu=LF_power / (Total_power-VLF_power)×100;

[0064] HF_nu=HF_power / (Total_power-VLF_power)×100.

[0065] (2) LF / HF ratio: LF / HF = LF_nu / HF_nu. When the LF / HF ratio increases, it indicates sympathetic dominance (such as in a stress state), while a decrease indicates parasympathetic dominance (such as in a relaxed state).

[0066] The loss function of the multimodal fusion evaluation model combines multi-task loss and regularization strategies:

[0067] import torch

[0068] import torch.nn as nn

[0069] class CustomLoss(nn.Module):

[0070] def_init_(self,alpha=0.7,beta=0.3):

[0071] super()._init_()

[0072] self.alpha = alpha # CLI task weight

[0073] self.beta = beta#ESS task weight

[0074] self.mse = nn.MSELoss()

[0075] self.kl_div = nn.KLDivLoss()

[0076] Loss setting and regularization strategy:

[0077] def forward(self, cli_pred, ess_pred, cli_true, ess_true):

[0078] loss_cli = self.mse(cli_pred, cli_true) # CLI regression loss (MSE)

[0079] loss_ess = self.mse(ess_pred, ess_true) # ESS classification loss (assuming segmented as -1 / 0 / +1)

[0080] total_loss = self.alpha * loss_cli + self.beta * loss_ess # Multi-task joint loss

[0081] total_loss += 0.01 * torch.norm(model.fusion_layer.weight, p=1) # Add feature sparsity constraint (L1 regularization)

[0082] return total_loss

[0083] In the above key settings, CLI (Cognitive Load Index) output: use MSE loss, cognitive load takes continuous value [0, 1] (0 = low load, 1 = high load), the larger the value, the heavier the mental burden.

[0084] ESS (Emotional Valence Score (or Emotional State Score)) output: use Tanh activation output [-1, 1] interval, -1 represents negative emotion, +1 represents positive emotion.

[0085] Dynamic weight (optional): the above α is 0.7, β is 0.3, in actual application scenarios, α / β can be adjusted according to the importance of the task (such as setting α=0.8 when clinical attention is paid to CLI).

[0086] Further, the model training process can be optimized in the following way as shown in Table 1.

[0087] Table 1

[0088]

[0089]

[0090] Application scenario examples:

[0091] Real-time feedback: when CLI>0.7 and ESS<-0.5, the VR system automatically switches to the decompression scene.

[0092] Long-term tracking: through the time correlation of LF / HF ratio and MoCA score, the training effect is evaluated.

[0093] The architecture balances the physiological signal time sequence characteristics and the scale static characteristics, realizes efficient joint modeling through multi-task learning, and is suitable for resource-limited embedded VR system deployment.

[0094] Next, combined with the above obtained scoring scheme, the implementation of the training process will be described.

[0095] Figure 5 is a flowchart showing a cognitive training method according to the present application.

[0096] As shown in Figure 5 , at step S501, the cognitive load index and emotional stability score are obtained.

[0097] At step S502, a corresponding training scheme is generated based on the cognitive load index and emotional stability score.

[0098] At step S503, the θ / α power ratio and the ratio of low frequency power to high frequency power are obtained in real time, and the θ / α power ratio and the ratio of low frequency power to high frequency power are combined to perform environmental simplification and / or decompression intervention. In some embodiments, the power features and reaction time are compared with the corresponding threshold values to determine the cognitive state. The power features include the θ / α power ratio, and the reaction time includes the state duration. The cognitive overload state can be determined according to the θ / α power ratio being greater than a set threshold value and the state duration being greater than a set time length, so as to determine the cognitive state. For example, the set threshold value can be 1.8 times the baseline standard deviation, and the set time length can be 8 seconds. After the cognitive state is determined, the corresponding environmental simplification and / or decompression intervention measures are combined to implement the training process.

[0099] In some embodiments, the environmental simplification includes three levels of environmental simplification:

[0100] In response to θ / α power ratio∈[1.5,2) or LF / HF∈[3,4), primary simplification is performed, and VR particle special effects are turned off and target highlights are enhanced; LF / HF is the ratio of low frequency power to high frequency power.

[0101] Intermediate simplification, 3D scene reduced to 2D, task target number halved, in response to θ / α power ratio ∈ [2, 2.5) or LF / HF ∈ [4, 5).

[0102] Full simplification, switch to static background, enable automatic aiming and breathing guidance animation, in response to θ / α power ratio ∈ [2.5, ∞) or LF / HF ∈ [5, ∞).

[0103] Embodiment 1:

[0104] If EEG detects a sudden surge in theta wave power (cognitive overload), automatically simplify the complexity of the virtual environment. At this time, the judgment condition of EEG signal (sudden surge in theta wave power) is: ① The theta wave (4-7 Hz) power value of frontal lobe lead (Fp1 / Fp2) > 1.8 times the standard deviation of the user's resting baseline value (dynamic calibration). ② The θ / α power ratio is continuously > 1.5 for more than 8 seconds (anti-mis touch). Maintain hardware synchronization during detection: mark EEG events with TTL pulses, aligned with VR engine (Unity / Unreal) timestamps.

[0105] Embodiment 2:

[0106] If ECG shows a decrease in HRV (stress rising), trigger a deep breathing guidance animation. At this time, the judgment condition of ECG signal (decrease in HRV) is: ① LF / HF ratio > 3.0 and SDNN (RR interval standard deviation) drops > 15%. ② Combine EDA (skin electricity) signal to verify stress state.

[0107] The following Table 2 shows the trigger conditions, specific operations and technical implementation methods corresponding to several simplification levels:

[0108] Table 2

[0109]

[0110] The following is part of the core program content:

[0111]

[0112]

[0113] User wears VR headset with integrated eye tracking, EEG cap, wristband ECG, and force feedback handle during training. The system issues evaluation instructions, and the user performs traditional cognitive scale evaluation. Feature extraction and fusion are performed, and the model outputs cognitive load index (CLI) and emotional stability score (ESS). The system recommends game training based on evaluation results, and collects handle operation accuracy (error <2 cm effective) and EEG theta / beta power ratio (ADHD threshold >4.5) for training analysis. The system records data: evaluation data, training results, user information, etc.

[0114] Taking the "insect catching" task as an example, the user wears the device to start the VR "insect catching" task. The system detects a decrease in alpha wave power (distraction) and increases the flicker frequency of the target objects in the game to attract attention. After training, the report shows that the concentration has increased by 20% (alpha wave power change) and the task completion speed has increased by 15% (behavior data).

[0115] The feature fusion algorithm used in the application is:

[0116] CLI = w_1 * MoCA + w_2 * (θ / α) + w_3 * P300 + w_4 * LF / HF

[0117] ESS = v_1 * MoCA_{emotion sub-item} + v_2 * (θ / α) + v_3 * LF / HF

[0118] The core of this algorithm is multi-modal information fusion. It does not rely on a single data source (such as only using scales or only using EEG), but combines the information of three dimensions: subjective behavior performance (MoCA), central nervous physiological activity (θ / α, P300), and peripheral autonomic nervous activity (LF / HF) through weighted linear combination. This fusion can offset the noise and limitations of a single modality, resulting in a more comprehensive, robust, and accurate cognitive-emotional state evaluation result (CLI and ESS) than any single indicator.

[0119] Weight determination: Use Ridge Regression to fit the clinical evaluation gold standard data. Example weights (validated by ADHD data set): CLI weights: w = [0.4, 0.3, 0.2, 0.1]. ESS weights: v = [0.5, 0.2, 0.3].

[0120] Verification case:

[0121]

[0122] Model accuracy:

[0123] CLI classification accuracy: 87.3% (k-fold validation). ESS correlation with clinical depression scale (HADS): r = 0.81. Therefore, according to the scheme of the present application, the accuracy of the assessment of cognitive ability is effectively improved, and the effect of cognitive ability training is guaranteed.

[0124] Figure 6 is a schematic diagram showing the constituent structure of an electronic device according to the present application.

[0125] As Figure 6 indicated, the present application also provides an electronic device comprising a processor and a memory, wherein the memory stores executable instructions of the processor, and when the processor executes the executable instructions, a cognitive ability assessment method as described above is implemented, or a cognitive training method as described above is implemented.

[0126] Further, the present application also provides a computer readable storage medium, wherein a computer program is stored on the computer readable storage medium, and when the computer program is executed by a processor, a cognitive ability assessment method as described above is implemented, or a cognitive training method as described above is implemented.

[0127] In the present application, the aforementioned memory can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus or device. For example, the computer readable storage medium can be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high bandwidth memory (HBM), hybrid memory cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, a module or both. Any such computer storage medium can be part of the device or accessible or connectable to the device. Any application or module described in the present application can be implemented using computer readable / executable instructions that can be stored or otherwise held by such a computer readable medium.

[0128] While the specification has illustrated and described various embodiments of the application, it will be clear to those of ordinary skill in the art that various changes, modifications, and substitutions can be made thereto without departing from the spirit and scope of the application. It is understood that in the process of practicing the application, various alternatives, modifications, and equivalents can be employed.

Claims

1. A VR-based cognitive ability assessment and training system, characterized in that, include: The data acquisition unit is used to collect the user's electrophysiological signal data and behavioral data; The assessment unit, which is connected to the data acquisition unit, is used to perform cognitive assessment scale tasks and simultaneously acquire the user's electrophysiological signal data, behavioral data, and scale scores. The fusion analysis unit, connected to the data acquisition unit and the evaluation unit, is used to integrate cognitive assessment scale data, electrophysiological signals and behavioral data and input them into the trained multimodal fusion assessment model to output cognitive load index and emotion stability score. The training unit, connected to the fusion analysis unit, is used to adjust the VR training difficulty based on the cognitive load index and emotional stability score.

2. The VR-based cognitive ability assessment and training system according to claim 1, characterized in that, Electrophysiological signal data include EEG and ECG signals, and behavioral data includes reaction time.

3. The VR-based cognitive ability assessment and training system according to claim 1, characterized in that, The evaluation unit is also configured to: Bandpass filtering and noise reduction are performed on electrophysiological signals; The power spectral density of the denoised electrophysiological signal was calculated and the frequency bands were divided to calculate the θ / α power ratio and the ratio of low-frequency power to high-frequency power.

4. A method for assessing cognitive abilities, characterized in that, The method applied to the VR-based cognitive ability assessment and training system according to any one of claims 1-3, the method comprising: Present cognitive scale tasks in a VR environment; Perform cognitive assessment scale tasks and simultaneously acquire users' electrophysiological signal data and behavioral data; The cognitive assessment scale data, electrophysiological signals, and behavioral data are integrated and input into a trained multimodal fusion assessment model to output a cognitive load index and an emotion stability score.

5. The cognitive ability assessment method according to claim 4, characterized in that, Integrating cognitive assessment scale data, electrophysiological signals, and behavioral data, including: Extract scores from the cognitive assessment scale task; Electrophysiological signals are preprocessed and divided into frequency bands to extract EEG and ECG signal features. The EEG signal features include the θ / α power ratio, and the ECG signal features include the HRV frequency domain index.

6. The cognitive ability assessment method according to claim 5, characterized in that, Electrophysiological signals are preprocessed and divided into frequency bands to extract EEG and ECG signal features, including: Bandpass filtering and noise reduction are performed on electrophysiological signals; The power spectral density of the denoised electrophysiological signal was calculated and the frequency bands were divided to calculate the θ / α power ratio and HRV frequency domain index.

7. A cognitive training method, characterized in that, The method applied to the VR-based cognitive ability assessment and training system according to any one of claims 1-3, the method comprising: Obtain cognitive load index and emotional stability score; Training plans are generated based on cognitive load index and emotional stability score; The system acquires the θ / α power ratio and the ratio of low-frequency power to high-frequency power in real time, and performs environmental simplification and / or stress reduction intervention based on the θ / α power ratio and the ratio of low-frequency power to high-frequency power.

8. The cognitive training method according to claim 7, characterized in that, The environment simplification includes three levels of environment simplification: In response to the θ / α power ratio ∈ [1.5,2) or LF / HF ∈ [3,4), a primary simplification is performed, VR particle effects are turned off, and target highlighting is enhanced; LF / HF is the ratio of low-frequency power to high-frequency power; In response to the θ / α power ratio ∈ [2,2.5) or LF / HF ∈ [4,5), intermediate simplification is performed, reducing the 3D scene to 2D and halving the number of task objectives; In response to θ / α power ratio ∈ [2.5,∞) or LF / HF ∈ [5,∞), perform full simplification, switch to static background, enable auto-aiming and breathing guidance animation.

9. An electronic device, characterized in that, include: processor; as well as The memory stores executable instructions of the processor, which, when executed by the processor, implement a cognitive ability assessment method as described in claims 4-6, or a cognitive training method as described in claim 7 or 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements a cognitive ability assessment method as described in claims 4-6, or a cognitive training method as described in claim 7 or 8.

Images