Spatial cognitive function training and evaluation method and system based on virtual reality

Abstract

The present invention discloses a spatial cognitive function training and evaluation method and system based on virtual reality, which relates to the field of cognitive function training and electroencephalogram signal processing technology. Based on virtual reality technology, VCM pre-test experiment, VRCYM training experiment and VCM post-test experiment of spatial cognitive function are sequentially performed on patients with cognitive impairment, and the time and path map of each experiment are recorded. The EEG data before and after each decision of the patient in each training experiment are obtained by using brain-computer interface technology; statistical analysis is performed according to the recorded data to obtain the behavioral data index value; time-frequency feature extraction and analysis are performed on the obtained EEG data to obtain the brain region information flow index value and the significant difference index value of the same brain region; according to the behavioral data index value, the brain region information flow index value and the brain region significant difference index value, the spatial cognitive function changes of the patient before and after training are comprehensively evaluated. The present invention can train and accurately evaluate the spatial cognitive function of patients.

Description

Technical Field

[0001] The present invention relates to the field of cognitive function training and electroencephalogram signal processing technology, and in particular to a method and system for spatial cognitive function training and evaluation based on virtual reality. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Patients with cognitive impairment can be divided into subjective cognitive decline (SCD), mild cognitive impairment (MCI) and Alzheimer's disease (AD) according to the different stages of cognitive impairment. SCD and MCI patients have a slight decline in cognitive function, but have not yet reached the severity of dementia. If not corrected in time, there will be a risk of developing Alzheimer's disease or other types of dementia. At present, there are two main interventions for SCD and MCI patients: cognitive training and drug therapy. Compared with drug therapy, cognitive training has the advantages of no side effects and is more effective. Spatial cognitive function is a high-level cognitive function, including spatial perception, spatial imagination, spatial memory, spatial reasoning and other abilities. This function can help people navigate, judge the relationship between objects, plan paths, and make decisions in complex environments in daily life. Therefore, the training of spatial cognitive function is particularly important for SCD and MCI patients.

[0004] Traditional spatial cognitive function training for patients with cognitive impairment usually relies on the patient's own imagination and the use of color cards or scene pictures to assist memory. The whole process requires guiding the patient to build a mental map by himself. With the rapid development of neuroscience and cognitive science, more and more studies have shown that brain plasticity plays a key role in the recovery and enhancement of cognitive function, especially in the early stages of cognitive impairment. Appropriate training and intervention can effectively delay or even reverse the decline of cognitive function. Using virtual reality (VR) to build training scenes for training can reactivate or improve multiple cortical functions of the brain and help optimize the encoding efficiency of the sensory cortex. Due to its immersive experience and high interactivity, VR technology has become one of the important tools in cognitive function training. In addition, the development of brain-computer interface (BCI) technology provides new possibilities for cognitive function assessment and personalized training. BCI technology can monitor and analyze patients' brain activity in real time, helping to identify cognitive load and attention level during training. At present, some studies have combined VR with BCI for rehabilitation training of patients with cognitive impairment, but the existing training methods based on the combination of VR and BCI still have certain shortcomings, as follows:

[0005] (1) The design of training scenarios is often relatively simple and lacks theoretical support from cognitive science experiments. Simple training scenarios, to a certain extent, limit the comprehensive training of patients’ spatial cognitive functions. The real-world environment is often complex and changeable, with multiple sensory stimulations and dynamic changes. In the development process, the existing VR training system relies more on technical implementation and application requirements, and less on the theoretical basis and experimental data of cognitive science. This design method lacking theoretical support may lead to a mismatch between the training content and the actual cognitive function requirements, and fail to fully exert the training effect.

[0006] (2) It is impossible to effectively process electroencephalogram (EEG) data and effectively evaluate whether the brain's cognitive function has improved before and after training. Currently, EEG data obtained through brain-computer interfaces are usually used to evaluate changes in brain cognitive function before and after training. However, traditional EEG data analysis methods are often unable to effectively capture subtle improvements in cognitive function. The temporal and spatial resolution of EEG data is limited and is easily affected by noise and artifacts, which makes it inaccurate to rely solely on simple band analysis or power spectral density to evaluate the effect of cognitive function improvement. In addition, traditional EEG data analysis methods often only focus on data at a single time point, while ignoring the complex temporal dynamic relationship and information flow direction between brain regions, which makes the evaluation of training effects lack depth and accuracy. Summary of the invention

[0007] In order to address the deficiencies of the above-mentioned prior art, the present invention provides a spatial cognitive function training and evaluation method and system based on virtual reality, which adopts a combination of virtual reality and brain-computer interface technology. While providing an immersive spatial cognitive function training scene environment, brain-computer interface technology is used to accurately monitor the patient's brain area activities. By integrating behavioral data and electroencephalogram signals, the changes in the patient's brain area's spatial cognitive function before and after training are accurately evaluated, thereby effectively improving the patient's training effect.

[0008] In a first aspect, the present invention provides a spatial cognitive function training and evaluation method based on virtual reality.

[0009] A spatial cognitive function training and evaluation method based on virtual reality, comprising:

[0010] Based on virtual reality technology, patients with cognitive impairment were subjected to VCM pre-test experiments, VRCYM training experiments, and VCM post-test experiments on spatial cognitive function. The time and path map of each experiment were recorded, and brain-computer interface technology was used to obtain EEG data before and after each decision of the patient in each training experiment.

[0011] Statistical analysis was performed based on the recorded time and path diagram to obtain behavioral data index values;

[0012] Each EEG data is preprocessed, and then the time-frequency features are extracted using the MST algorithm. A vector autoregression model is constructed based on the time-frequency features, and Granger causality analysis is performed to determine the direction and intensity of information flow, and then the information flow index value of the brain area is obtained;

[0013] According to the time-frequency characteristics of the EEG data obtained from the pre- and post-training experiments, the significant difference index values ​​of the same brain region were obtained through paired sample t-test;

[0014] Based on the behavioral data index values, brain area information flow index values, and brain area significant difference index values, the changes in patients' spatial cognitive function before and after training were comprehensively evaluated.

[0015] In a second aspect, the present invention provides a spatial cognitive function training and evaluation system based on virtual reality.

[0016] A spatial cognitive function training and evaluation system based on virtual reality, comprising:

[0017] input devices for triggering decision responses;

[0018] VR head-mounted display device, used to provide virtual training scenes;

[0019] Behavioral data processing and visualization equipment, used to record the time and path diagram for completing each experiment, and perform statistical analysis to generate visual images;

[0020] EEG acquisition equipment, used to collect and obtain EEG data of patients before and after each decision in each training experiment;

[0021] The portable electroencephalogram (EEG) signal processing and visualization device is used to execute the EEG data processing steps of the spatial cognitive function training and evaluation method based on virtual reality proposed in the first aspect, and generate a visualized brain network diagram.

[0022] In a third aspect, the present invention further provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned virtual reality-based spatial cognitive function training and evaluation method when executing the executable instructions stored in the memory.

[0023] In a fourth aspect, the present invention further provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned virtual reality-based spatial cognitive function training and evaluation method.

[0024] In a fifth aspect, the present invention also provides a computer program product, which includes executable instructions stored in a computer-readable storage medium; wherein, when a processor of an electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the above-mentioned virtual reality-based spatial cognitive function training and evaluation method is implemented.

[0025] One or more of the above technical solutions have the following beneficial effects:

[0026] 1. The present invention provides a method and system for spatial cognitive function training and evaluation based on virtual reality. It adopts a combination of virtual reality and brain-computer interface technology. While providing an immersive spatial cognitive function training scene environment, it uses brain-computer interface technology to accurately monitor the patient's brain area activities. By integrating behavioral data and electroencephalogram signals, it accurately evaluates the changes in the patient's brain area's spatial cognitive function before and after training, effectively improving the patient's training effect.

[0027] 2. The VRCYM training experiment in the present invention is designed and proposed based on the traditional cognitive Y-maze experiment. Through virtual reality technology, multiple Y-mazes are connected in a ring to form a ring-shaped continuous maze structure. By constructing this complex maze structure, the spatial cognitive ability of the subjects can be more effectively challenged, which helps to improve the patients' spatial perception and path planning ability in the actual environment.

[0028] 3. The present invention proposes a new EEG data analysis method, which can effectively capture the subtle improvement of cognitive function. This method can effectively remove artifacts in EEG by dividing EEG data into multiple different frequency bands for filtering, and then extract the time-frequency characteristics of the pre-processed EEG data, construct a vector autoregression model and perform Granger causality analysis, taking into account the complex temporal dynamic relationship between brain regions and the direction and intensity of information flow, which can ensure the depth and accuracy of the evaluation of training effects; in addition, the behavioral data indicators and EEG signals are combined for evaluation, and the changes in the spatial cognitive function of patients before and after training are comprehensively analyzed. By combining these two types of data, the effect of training can be evaluated more comprehensively, which can not only quantify the external manifestations of cognitive function, but also deeply understand the neural mechanism behind the improvement of cognitive function. This multi-dimensional evaluation method helps to more accurately capture the changes in cognitive function and ensure accurate evaluation and feedback of training effects.

[0029] 4. The present invention also proposes a VCM test experiment designed with horror as the theme, which uses Unity's URP (Universal Rendering Pipeline) to render the environment of a virtual corridor maze in real time. By enhancing light and shadow effects, adding dynamic shadows and high-quality material performance, a realistic and oppressive horror atmosphere is rendered. The horror atmosphere effectively simulates the patient's cognitive response under high pressure or complex situations, so as to improve the patient's immersion and concentration, enhance the task's involvement and improve the accuracy of the experimental data.

[0030] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0032] Figure 1 This is an overall flow chart of the spatial cognitive function training and evaluation method based on virtual reality according to an embodiment of the present invention;

[0033] Figure 2 This is a flow chart of performing spatial cognitive function training based on virtual reality in an embodiment of the present invention;

[0034] Figure 3 Schematic diagram of a pre-VCM test experiment in an embodiment of the present invention;

[0035] Figure 4 Schematic diagram of selecting the starting point and the ending point in the VRCYM training experiment in an embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of collecting EEG data during the VRCYM training experiment in an embodiment of the present invention;

[0037] Figure 6 This is a flow chart of evaluating spatial cognitive function based on brain-computer interface technology in an embodiment of the present invention;

[0038] Figure 7 Schematic diagram of EEG information flow vector generated based on EEG data in an embodiment of the present invention;

[0039] Figure 8 Schematic diagram of performing a t-test on EEG data before and after a training experiment in an embodiment of the present invention;

[0040] Fig. 9 4 is a block diagram of a spatial cognitive function training and evaluation system based on virtual reality in an embodiment of the present invention. DETAILED DESCRIPTION

[0041] It should be noted that the following detailed descriptions are exemplary only, are intended to describe specific embodiments, are intended to provide further explanation of the present invention, and are not intended to limit exemplary embodiments according to the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those of ordinary skill in the art to which the present invention belongs. In addition, it should also be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0042] Embodiment 1

[0043] This embodiment provides a spatial cognitive function training and evaluation method based on virtual reality. Figure 1 As shown, the specific steps include:

[0044] Step S1, based on virtual reality technology, conduct VCM (Virtual Corridor Maze) pre-test experiment, VRCYM (Virtual Ring Continuous Y-Maze) training experiment, and VCM post-test experiment of spatial cognitive function on patients with cognitive impairment in sequence, record the completion time and path map of each experiment, and use brain-computer interface technology to obtain EEG data of patients before and after each decision in each training experiment.

[0045] like Figure 2 As shown, firstly, virtual reality technology is used to train the spatial cognitive function of patients with cognitive impairment, including:

[0046] Step S1.1: Before the formal training begins, the patient first undergoes a VCM test experiment, namely a VCM pre-test experiment. Figure 3 As shown in the figure, the patient puts on a VR headset and enters a virtual corridor maze scene. First, a dynamic navigation path is created using Unity's NavigationMesh technology to guide the patient to familiarize himself with the path to the designated location of the maze. The whole process is slow so that the patient can recognize and remember the path. Then, all guidance is removed, allowing the patient to explore on his own with his own spatial cognitive ability. When the patient is sure that he has reached the designated location, the experiment ends. During the entire test experiment, the time it takes for the patient to complete the experiment and the path map of the self-exploration are recorded.

[0047] Preferably, a horror-themed VCM test experiment is designed, and Unity's URP (Universal Rendering Pipeline) is used to render the environment of a virtual corridor maze in real time. By enhancing light and shadow effects, adding dynamic shadows and high-quality material performance, a realistic and oppressive horror atmosphere is rendered, thereby simulating the patient's cognitive response under high pressure or complex situations, so as to improve the patient's immersion and concentration.

[0048] Step S1.2: Conduct a 15-day VRCYM training experiment on the patient. Specifically, the maze is set to have 30 rectangular platforms, such as Figure 4 As shown in the figure, before the experiment begins, a platform is randomly selected as the starting point, and a line is connected using the center point of the maze and the determined starting point. The rectangular platform inside and outside the ring of the connecting line is randomly determined as the ending point. After the starting point and the ending point are determined, the patient puts on the VR headset and the EEG acquisition cap (i.e., the EEG acquisition device) to enter the virtual ring continuous Y maze scene. Figure 5 As shown in the figure, first, a dynamic navigation path is created using Unity's Navigation Mesh technology to guide patients to familiarize themselves with the path to the designated location of the maze. All forks in the entire maze are equipped with landmarks (such as benches, flower pots, road signs, etc.). In addition, a central building group is placed in the middle of the maze. When patients are familiar with the path, they can use the above items to assist in spatial memory; then, all guidance is removed, allowing patients to explore on their own with their own spatial cognitive ability. When the patient determines that they have reached the designated terminal platform, the experiment ends. During the entire training experiment, the EEG signal data of the patient at each decision was recorded, and the time the patient completed the experiment and the path map of the self-exploration during the experiment were also recorded.

[0049] In the above process, the EEG acquisition device is used to collect the EEG signals of patients with cognitive function defects during training. In this embodiment, a 64-channel EEG signal acquisition system is used to collect the EEG signal data of the patient at a sampling frequency of 1000Hz. Among them, the position of each electrode in the EEG signal (EEG) acquisition device represents a node in the brain network, and the number of electrodes is the number of nodes N. Considering that the frontal lobe, parietal lobe, temporal lobe, occipital lobe, etc. are the main areas (i.e., brain areas) for the brain to perform cognitive function-related tasks, in order to reduce the interference of irrelevant information, 24 electrodes are selected for EEG acquisition. In the 64-channel EEG signal acquisition system, the corresponding electrodes for EEG acquisition are: "Fp1", "Fp2", "F3", "F4", "Fz", "AF3", "AF4", "P3", "P4", "P7", "P8", "Pz", "CP1", "CP2", "CP3", "CP4", "T7", "T8", "Cz", "O1", "O2", "Oz", "C3", "C4".

[0050] In addition, in the above process, the patient's walking in the maze is realized by keyboard control, and the decision response is triggered by the direction keys of the keyboard. The keyboard sends the trigger signal to the data processing end, and the data processing end retains the EEG data of 1 second before and after the decision response according to the trigger signal, so as to realize the use of EEG acquisition equipment to collect EEG signals during decision-making. Specifically, the EEG signal is divided according to the decision mark recorded during the experiment. Considering that the subject's keyboard key response during VR training is used as a decision mark, the time window is set to 1 second before and after the keyboard key response. Then, considering that the keyboard key response is asynchronous, the EEG data in the time window around the button press (ie, the decision mark) is extracted. Among them, if two consecutive windows overlap in time, the two periods are merged.

[0051] Preferably, in this embodiment, step S1.2 is repeated 30 times, twice a day, for a total of 15 days.

[0052] Step S1.3, the patient performs another VCM test experiment (i.e., the post-VCM test experiment). Specifically, the pre-VCM test experiment scene is mirrored, and the experimental process is still consistent with step S1.1. After the patient undergoes the VRCYM training experiment in step S1.2, a VCM test experiment is performed again, and the test results of this time are compared with the VCM test results before the VRCYM training experiment, so as to better evaluate the effect of the training.

[0053] That is, after completing the above training steps, the spatial cognitive function is evaluated, including:

[0054] Step S2: Perform statistical analysis based on the recorded time and path diagram to obtain behavioral data indicator values.

[0055] Specifically, for the completion time and path diagram of each experiment obtained, the time and path length are plotted into a bar graph to intuitively represent the changes in the patient's behavioral data for each experiment. According to the changes in the length of the completion time of each experiment and its changing trend, the complexity of the path (such as whether there is a detour, whether the path is the shortest, etc.) and its trend, the behavioral data indicator values ​​of the training (such as difference, trend value, etc.) are obtained, which can be used to evaluate the changes in the patient's spatial cognitive function before and after training.

[0056] Step S3: pre-process each EEG data obtained, then use the MST algorithm to extract the time-frequency features, build a vector autoregression model based on the time-frequency features and perform Granger causality analysis to determine the direction and intensity of information flow, and then obtain the brain area information flow index value.

[0057] Step S3.1: pre-process each acquired EEG data.

[0058] In this embodiment, the EEG signal data at the time of decision-making (i.e., within a 1-second time window before and after the decision response) is extracted using an EEG acquisition device, and artifact elimination and baseline drift correction are performed on the EEG signals of all extracted windows. Specifically, for the EEG signal in each frequency band, a moving window is used to extract samples, with a window width of 2s, a step length of 1s, and an overlap rate of 50%; further, for the extracted EEG signal data, the overlapping data of the continuous windows are first merged, that is: if two continuous windows overlap in time, the overlapping data of the two continuous windows are merged; then, a fifth-order Butterworth bandpass filter is used to decompose the EEG data into seven different frequency bands of δ, θ, α1, α2, β1, β2, and γ, and the time series signal of each frequency band of the EEG data is extracted. By smoothing the frequency response of the signal, the EEG data of the target frequency band is effectively retained, and the noise and artifacts of the non-target frequency band are filtered out; baseline correction is performed on all extracted window data, and the EEG data of each frequency band is combined into a new time series to obtain a total of 7 frequency time series.

[0059] Step S3.2: Based on each EEG data after preprocessing, the MST algorithm is used to extract the time-frequency features, and a vector autoregression model is constructed according to the time-frequency features and Granger causality analysis is performed to determine the direction and intensity of information flow, and then obtain the brain area information flow index value.

[0060] like Figure 6 As shown in the figure, first, the MST algorithm is used to perform time-frequency processing on the preprocessed EEG data, the optimal energy concentration in the time-frequency domain is calculated, and the time-frequency features are constructed and extracted. Specifically, in order to better analyze the time-frequency characteristics of EEG data, the MST algorithm is used to perform time-frequency processing on the EEG data. Compared with the traditional ST algorithm, MST optimizes two adaptive parameters to obtain a suitable window width, thereby more accurately calculating the energy of high-frequency components.

[0061] In this embodiment, MST can be defined as:

[0062]

[0063] Where x(t) is the EEG signal, ξ is the translation factor, h(ξ-t,f) is the Gaussian window function used to control the position of the Gaussian function on the t time axis, f is the frequency, and j is the imaginary unit, which is defined as follows:

[0064]

[0065] Where σ(f) is the standard deviation of the Gaussian function and is defined as follows:

[0066]

[0067] Among them, MST can obtain the optimal energy concentration in the time-frequency domain by continuously adjusting the two parameters p and q. Specifically, by adjusting the p and q parameters, MST can control the degree of localization of the Gaussian window function in the time and frequency domains. When the frequency is high, the standard deviation is small, which means that the window is narrower, thereby improving the frequency resolution, but this may sacrifice the time resolution; similarly, when the frequency is low, the standard deviation is large, which means that the window is wider, thereby improving the time resolution, but it may sacrifice the frequency resolution. Therefore, in order to achieve the optimal energy concentration, it is necessary to select appropriate p and q parameters so that the energy of the signal is neither too dispersed nor too concentrated in the time-frequency plane, so that the signal characteristics can be more prominent.

[0068] Through the MST algorithm, the time-frequency features of the EEG data in the appropriate frequency band are extracted. The time-frequency features can be referred to as PSD features. The PSD features can be defined as:

[0069]

[0070] Secondly, the time-frequency characteristics of multiple brain regions are used to construct a multidimensional vector, and the Bayesian information criterion is used to select the optimal lag order to construct a vector autoregression model. Specifically, the extracted PSD eigenvalues ​​are used as time series to construct a multidimensional vector, which is:

[0071] X t =[PSD1(t), PSD2(t),...,PSD n (t)] T ;

[0072] Among them, PSD n (t) represents the characteristic value of the nth brain region (i.e., the area where the nth pole piece is located) at time t.

[0073] The Bayesian information criterion is used to select the optimal lag order. Based on the determined lag order p, a vector autoregression model, namely the VAR model, is constructed, which can be expressed as:

[0074] X t =c+A1X t-1 +A2X t-2 +…+A p X t-p +∈ t ;

[0075] In the formula, X t A represents the data vector at time t; i is the coefficient matrix; p is the lag order; c is the constant term matrix, representing the intercept of the model, and its form is the same as X t The constant is calculated by taking the average value of the sample data at all time points. The specific formula is:

[0076]

[0077] ∈ t is the noise term (also called the residual term), which is actually the model prediction value and the actual observed value X t The difference between represents the prediction error of the model at time t, which can be expressed as:

[0078]

[0079] Estimate the coefficient matrix A of each lag term by the least squares method i , by testing the independence and stability conditions of the residuals, the goodness of fit and predictive ability of the model are verified.

[0080] Afterwards, based on the vector autoregression model (VAR model) constructed above, a basic model containing only the lag term of the target brain region variable itself is constructed as follows:

[0081]

[0082] In fact, the above basic model is a rewritten form of the VAR model.

[0083] On the basis of the basic model, the lag term of the potential brain region variables is added to construct an extended model, which can be expressed as:

[0084]

[0085] The constant term c′ and the residual term ∈′ in the above extended model t The calculation method of is the same as the basic model, and the calculation formula of this constant term is:

[0086]

[0087] The calculation formula of the residual term is:

[0088]

[0089] The least squares method is also used to estimate the coefficient matrix of each lag term, and the independence and stability conditions of the residuals are tested to verify the goodness of fit and predictive ability of the model.

[0090] Then, the time series multidimensional vector values ​​of the target brain area and the affected brain area were substituted into the basic model and the extended model respectively, and the coefficient matrix of each lag term in the model was estimated by the least squares method, and then the constant term and residual term in the model were calculated.

[0091] In the above process, according to the medical division of brain regions and their functions, and according to actual needs, target brain regions and influencing brain regions are selected for brain region information flow analysis. Specifically, the brain region to be analyzed (such as analyzing which other brain regions will affect the brain region) is randomly selected as the target brain region, and the brain region with similar functions is selected as the influencing brain region. For example, in this embodiment, the prefrontal lobe (attention, decision-making) in cognition is used as the target brain region, and the parietal lobe (spatial perception, perception) with similar functions is used as the influencing brain region, and the influence of the parietal lobe on the prefrontal lobe and the brain region information flow are analyzed.

[0092] In this way, the direction of information flow between brain regions is first determined, and then the time series multidimensional vector values ​​of the target brain region and the affected brain region are obtained. The prefrontal lobe is represented by seven electrodes (Fp1, Fp2, F3, F4, Fz, AF3, AF4), and the PSD characteristics of all electrodes are measured to form a time series multidimensional vector X t , each vector is a column vector of 7 rows; similarly, the parietal lobe is covered by seven electrodes (P3, P4, Pz, CP1, CP2, CP3, CP4), and the PSD characteristics of all electrodes are measured to form a multidimensional vector X′ of the time series t , each vector is a column vector with 7 rows.

[0093] Then, the time series multidimensional vector values ​​of the target brain area obtained above are substituted into the basic model, and the matrix coefficient α is estimated and fitted using the least squares method. i , and then calculate the constant term and residual term in the model; further, the time series multidimensional vector value X of the target brain area t And the time series multidimensional vector value X′ affecting the brain area t Substitute it into the extended model, use the least squares method to estimate and fit the matrix coefficient β i , and then calculate the constant term and residual term in the model.

[0094] Finally, the residual term of the basic model is calculated Calculate the residual sum of squares RSS restricted ,for:

[0095]

[0096] Where T is the total number of observed samples.

[0097] Similarly, calculate the residual sum of squares RSS of the extended model unrestricted ,for:

[0098]

[0099] Based on the residual sum of squares of the basic model and the extended model, the F statistic used to test the significant difference between the two models is calculated as:

[0100]

[0101] Furthermore, in order to determine whether the calculated F statistic is significant, it is necessary to find the corresponding critical value in the F distribution table. The two degrees of freedom of the F distribution are: m, which is the number of new parameters in the extended model. Taking the extended model as an example, there are 3 brain regions that have an impact on the target brain region and the lag order of each brain region is 2, then the number of new parameters in the extended model is m=3×2=6; Tk is the total number of samples in the extended model minus the number of parameters.

[0102] Finally, the Granger causal relationship between the target brain region and other brain regions was analyzed based on the comparison of the F statistic and the critical value. It is assumed that the extended model significantly improves the prediction accuracy of the target brain region variables. If the F statistic is greater than the critical value, it means that the extended model is significantly better than the basic model, indicating that the lag terms of other brain region variables have a Granger causal relationship with the target brain region variables; conversely, if the F statistic is less than or equal to the critical value, the extended model is not significantly better than the basic model, and it cannot be concluded that other brain region variables have a Granger causal relationship with the target brain region variables.

[0103] like Figure 7 As shown in the figure, according to the above analysis, the target brain area and other brain areas that have an impact on the brain area are calculated, so that the direction of the information flow can be determined from the impacting brain area to the target brain area. At the same time, ln(F) is calculated as the strength of Granger causality (i.e., the strength of information flow), and Granger causality is converted into information flow vectors (including direction and size) between different brain areas, which are used to represent the information flow vectors between different brain areas and obtain the brain area information flow index value. Among them, when the calculated information flow intensity is almost 0, it is considered that there is no information flow between the two packets, that is, there is no mutual influence between the two brain areas.

[0104] Furthermore, the information flow vectors between different brain regions are marked in the brain network, and the brain network diagrams based on the information flow vectors of each training experiment are compared. According to the changes in the connection and strength of the information flow vectors between different brain regions, the connectivity and information transfer efficiency between brain regions are determined. For example, if the connection and strength of the information flow vectors between different brain regions of the patient increase, it means that this experiment has produced a positive effect on the patient's brain function training and enhanced the connectivity and information transfer efficiency between brain regions.

[0105] Step S4: Based on the time-frequency characteristics of the EEG data obtained from the pre- and post-training experiments, a paired sample t-test is performed to obtain the significant difference index value of the same brain region.

[0106] like Figure 8As shown, using the EEG data preprocessed in step S3, the time-frequency features of the EEG data obtained from the previous and next training experiments (such as the last experiment of the previous day and the first experiment of the day) are subjected to a paired sample t-test on the PSD features of each frequency band, for a total of 14 times. The average power difference (i.e., the average PSD difference) between the last experiment of the previous day and the first experiment of the day is calculated, and a statistical test is performed to evaluate whether the difference is significant.

[0107] Furthermore, the calculation method of the t-test in statistics is used to calculate the difference of each pair of samples i, and then calculate the mean and standard deviation of all the differences, and then calculate the t value, and then find the corresponding p value based on the t distribution. The p value reflects the probability of the observed difference occurring under the null hypothesis. Specifically, the formula for calculating the mean of the difference is:

[0108]

[0109] The standard deviation of the difference is calculated as:

[0110]

[0111] Then, the calculation formula of t value is:

[0112]

[0113] The t value and corresponding p value of each frequency band were calculated, where the t value reflects the degree of difference between the groups, and the p value is used to determine whether these differences are statistically significant. The Benjamini-Hochberg FDR correction method was used to perform FDR (false discovery rate) correction on the calculated p value to control the problem of multiple comparisons, thereby ensuring that the proportion of false positives in multiple tests remains at a reasonable level. The adjusted p value is used as the indicator value of significant differences between different brain regions to determine whether there are statistically significant differences. For example, when the p value is lower than the set significance level, it indicates that there is a significant difference in EEG activity before and after training.

[0114] Furthermore, by comparing the significant difference indicators of EEG activity in daily experiments, if the differences in the patient's EEG activity in daily experiments are significant, it means that this experiment has a positive impact on the patient's brain area activity.

[0115] Step S5: Comprehensively evaluate the changes in the patient's spatial cognitive function before and after training based on the behavioral data index value, the brain region information flow index value, and the brain region significant difference index value. Specifically, the brain region information flow index value and the brain region significant difference index value are mainly used, and the behavioral data index value is used as a supplement to conduct a comprehensive evaluation. The more complex the brain region information flow network and the more obvious the brain region significant difference, the better the training effect.

[0116] Embodiment 2

[0117] This embodiment provides a spatial cognitive function training and evaluation system based on virtual reality. Fig. 9 As shown, specifically including:

[0118] input devices for triggering decision responses;

[0119] VR head-mounted display device, used to provide virtual training scenes;

[0120] Behavioral data processing and visualization equipment, used to record the time and path diagram for completing each experiment, and perform statistical analysis to generate visual images;

[0121] EEG acquisition equipment, used to collect and obtain EEG data of patients before and after each decision in each training experiment;

[0122] The portable electroencephalogram (EEG) signal processing and visualization device is used to execute the EEG data processing steps of the spatial cognitive function training and evaluation method based on virtual reality proposed in the first embodiment, and generate a visualized brain network diagram.

[0123] Embodiment 3

[0124] The present embodiment provides an electronic device, comprising: an input device, a memory and a processor; the memory is used to store executable instructions; the processor is used to implement the above method provided in the present embodiment when executing the executable instructions stored in the memory.

[0125] The input device may be a keyboard or a handle, the processor may be a central processing unit (CPU) or a graphics processing unit (GPU), and the memory includes a read-only memory (ROM) and a random access memory (RAM), which are used to store relevant information.

[0126] Furthermore, multiple IPS screens are connected to the above-mentioned electronic device, and the electronic device is connected to the VR head display device. Different functions are displayed on each IPS screen, including the virtual scene screen during patient training, the real-time path map of the patient in the virtual scene, the statistical graph of the experimental time and path length for each completion, the real-time information flow vector brain network map and the significant difference indicator display area. According to the real-time changes of the virtual scene screen during patient training, the content on each IPS screen is refreshed to provide real-time human-computer interaction feedback. During the processing, according to the progress of the processing, the corresponding progress information is displayed on the IPS screen interface, such as the percentage of the processing progress. The patient's mobile control operation in the virtual scene is realized through the keyboard, and the operation information is stored in the memory for retaining the EEG signal of a specific time period.

[0127] Embodiment 4

[0128] This embodiment also provides a computer-readable storage medium storing executable instructions. When the executable instructions are executed by a processor, the processor will be caused to execute the above method provided in this embodiment.

[0129] Embodiment 5

[0130] This embodiment provides a computer program product, which includes an executable instruction, which is a computer instruction; the executable instruction is stored in a computer-readable storage medium. When a processor of an electronic device reads the executable instruction from the computer-readable storage medium and the processor executes the executable instruction, the electronic device executes the above method provided in this embodiment.

[0131] The steps involved in the above embodiments 2 to 5 correspond to the method embodiment 1. For the specific implementation, please refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0132] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0133] The above description is only a preferred embodiment of the present invention. Although the specific implementation mode of the present invention is described in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present invention, various modifications or deformations that can be made by technical personnel in this field without creative work are still within the scope of protection of the present invention.

Claims

1. A spatial cognitive function training and evaluation method based on virtual reality, characterized in that: include: Based on virtual reality technology, patients with cognitive impairment were subjected to VCM pre-test experiments, VRCYM training experiments, and VCM post-test experiments on spatial cognitive function. The time and path map of each experiment were recorded, and brain-computer interface technology was used to obtain EEG data before and after each decision of the patient in each training experiment. Statistical analysis was performed based on the recorded time and path diagram to obtain behavioral data index values; Each EEG data is preprocessed, and then the time-frequency features are extracted using the MST algorithm. A vector autoregression model is constructed based on the time-frequency features, and Granger causality analysis is performed to determine the direction and intensity of information flow, and then the information flow index value of the brain area is obtained; According to the time-frequency characteristics of EEG data obtained from the pre- and post-training experiments, the significant difference index values ​​of the same brain region were obtained through paired sample t-test; Based on the behavioral data index values, brain area information flow index values, and brain area significant difference index values, the changes in patients' spatial cognitive function before and after training were comprehensively evaluated.

2. A method for spatial cognitive function training and evaluation based on virtual reality as claimed in claim 1, characterized in that: Each EEG data is preprocessed, including: For all extracted EEG data, overlapping data of continuous windows are merged; Artifact removal is performed on each EEG data by: using a bandpass filter to decompose the EEG data into signals of multiple frequency bands, extracting the time series signal of each frequency band of the EEG data, smoothing the frequency response of the signal, retaining the EEG data of the target frequency band and filtering out the noise and artifacts of the non-target frequency band; The EEG data is corrected for baseline drift, which is: baseline correction is performed on all extracted window data, and the EEG data of each frequency band is combined into a new time series to obtain a time series of multiple frequencies.

3. A method for spatial cognitive function training and evaluation based on virtual reality as claimed in claim 1, characterized in that: The determination of the brain region information flow index value includes: The preprocessed EEG data were processed in time and frequency using the MST algorithm to calculate the optimal energy concentration in the time and frequency domain, and to construct and extract time and frequency features; The time-frequency characteristics of multiple brain regions are used to construct a multidimensional vector, and the Bayesian information criterion is used to select the optimal lag order to build a vector autoregressive model. According to the vector autoregression model, a basic model containing only the lag term of the target brain region variable itself was constructed, and the lag term of the potential brain region variable was added to the basic model to construct an extended model; Substitute the time series multidimensional vector values ​​of the target brain area and the affected brain area into the basic model and the extended model respectively, use the least squares method to estimate the coefficient matrix of each lag term in the model, and then calculate the constant term and residual term in the model; According to the residual sum of squares of the basic model and the extended model, the F statistic used to test the significant difference between the two models was calculated. According to the comparison between the F statistic and the critical value, the Granger causality between the target brain area and other brain areas was analyzed, and the Granger causality was converted into an information flow vector between different brain areas to obtain the brain area information flow index value; the information flow vector includes the information flow direction and information flow intensity.

4. A method for spatial cognitive function training and evaluation based on virtual reality as claimed in claim 3, characterized in that: The information flow vectors between different brain regions are marked in the brain network, and the brain network diagram based on the information flow vector of each training experiment is compared. According to the changes in the connection and strength of the information flow vectors between different brain regions, the connectivity and information transfer efficiency between brain regions are determined.

5. A method for spatial cognitive function training and evaluation based on virtual reality as claimed in claim 1, characterized in that: The determination of the significant difference index value of the same brain region includes: According to the time-frequency characteristics of the EEG data obtained from the pre- and post-training experiments, a paired sample t-test was performed on the time-frequency characteristics of each frequency band, and the t value and corresponding p value of each frequency band were calculated; The calculated p-values ​​were corrected for false discovery rate to obtain the significant difference index values ​​between different brain regions.

6. A method for spatial cognitive function training and evaluation based on virtual reality as claimed in claim 5, characterized in that: Compare the corrected p-value with the significance level setting value, and determine whether there is a significant difference before and after the training experiment based on the comparison results.

7. A spatial cognitive function training and evaluation system based on virtual reality, characterized in that: include: input devices for triggering decision responses; VR head-mounted display device, used to provide virtual training scenes; Behavioral data processing and visualization equipment, used to record the time and path diagram for completing each experiment, and perform statistical analysis to generate visual images; EEG acquisition equipment, used to collect and obtain EEG data of patients before and after each decision in each training experiment; A portable electroencephalogram (EEG) signal processing and visualization device, used to execute the EEG data processing steps of the spatial cognitive function training and evaluation method based on virtual reality as described in any one of claims 1 to 6, and generate a visualized brain network diagram.

8. An electronic device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the spatial cognitive function training and evaluation method based on virtual reality as described in any one of claims 1 to 6 when executing the executable instructions stored in the memory.

9. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause the processor to execute the executable instructions to implement the spatial cognitive function training and evaluation method based on virtual reality as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product includes executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the spatial cognitive function training and evaluation method based on virtual reality as described in any one of claims 1 to 6 is implemented.

Images