Mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram
By combining virtual reality tests and EEG data, using a mild cognitive impairment diagnosis system of virtual reality and EEG, the problem that the existing technology cannot effectively reflect the brain's cognitive status and high equipment costs is achieved, and mild Rapid, accurate diagnosis and popularity of cognitive impairment.
Patent Information
- Application Number
- CN202410759029.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-13
- Publication Date
- 2024-11-15
AI Technical Summary
The prior art cannot effectively reflect the cognitive status of the brain, and the multi-channel demand of EEG devices limits its popularity in the diagnosis of mild cognitive impairment.
Using a mild cognitive impairment diagnosis system based on virtual reality and EEG, the virtual reality test module, data preprocessing module and voting strategy-based decision fusion module combine virtual reality test and EEG data to achieve the test Two-dimensional diagnosis of cognitive performance and brain state.
It realizes rapid and accurate diagnosis of mild cognitive impairment, reduces equipment costs and diagnosis time, and improves the reliability and popularity of diagnosis.
Smart Images

Figure CN118969238A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to mild cognitive impairment diagnosis and wearable device technology in the field of neuropsychology, and is specifically a mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram. Background Art
[0002] As the aging of my country's population continues to deepen, the health of the elderly faces serious threats, among which Alzheimer's disease (AD) is particularly prominent. AD is a complex neurodegenerative disease. Once patients develop clinical symptoms, it is difficult to reverse. Mild cognitive impairment is a cognitive state between normal aging and early AD, with a certain degree of reversibility and uncertainty. This stage will last for 15-25 years, providing the best window period for the prevention and treatment of AD. However, the current diagnostic methods for mild cognitive impairment used in clinical practice cannot meet the needs of large-scale screening.
[0003] Virtual reality (VR) technology has shown great potential in the diagnosis and screening of cognitive impairment. VR technology creates a fully immersive virtual environment and designs gamified psychophysical tasks. The VR environment enables researchers to assess subjects' cognitive impairment in a completely new way. These assessments provide a testing environment that is closer to real-life scenarios and enable cognitive impairment assessments based on subjects' task completion scores and other behavioral characteristics. However, despite the significant advantages of VR technology in the diagnosis and screening of cognitive impairment, it still faces some challenges. Current VR methods mainly focus on assessing behavioral responses, which cannot fully reflect the cognitive state of the brain and effectively distinguish cognitive impairment.
[0004] As a simple, non-invasive neuroimaging tool, electroencephalogram (EEG) plays an increasingly important role in the screening of cognitive impairment. By measuring the brain's electrical activity, EEG can directly reflect the state of brain waves at the synaptic level. Many studies at home and abroad have demonstrated the potential of EEG in the diagnosis of cognitive impairment. However, its widespread use is limited by its reliance on large multi-channel equipment, which limits the popularity of EEG in clinical and community settings.
[0005] VR has been shown to be able to develop new screening tools for altering mental states. VR cognitive tasks can be used as an effective way to induce cortical movement in patients with mild cognitive impairment, thereby reflecting their cognitive state and achieving a more accurate assessment of mild cognitive impairment. Summary of the invention
[0006] The technical problem to be solved by the present invention is to provide a mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram (EEG) in response to the deficiencies of the prior art. This system solves the problems existing in the current screening of mild cognitive impairment, effectively solves the problems that virtual reality cannot reflect the state of the brain, and that cognitive screening relies on multi-channel EEG equipment, thereby achieving effective diagnosis of mild cognitive impairment.
[0007] The technical solution of the present invention is as follows:
[0008] A mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, including a virtual reality test module, a data preprocessing module, and a decision fusion module based on voting strategy:
[0009] Virtual reality test module: This module contains targeted virtual reality tests, including attention test, depth perception test and working memory test. It uses immersive virtual reality technology to stimulate the subject's attention, depth perception ability and working memory cognitive fields respectively, and collects the subject's cognitive performance and EEG data in the process, so as to diagnose the subject's mild cognitive impairment from the two dimensions of cognitive performance and brain state; at the same time, the virtual reality test collects the start and end time of each test, so as to segment the EEG data of each test in the subsequent processing;
[0010] Data preprocessing module: After the test, the cognitive performance and EEG data of the virtual reality test are preprocessed to identify and remove possible outliers and erroneous data to ensure the reliability and accuracy of the data;
[0011] The decision fusion module based on voting strategy includes feature extraction, feature selection, feature fusion and decision fusion; feature extraction is used to extract features with strong representation ability of cognitive performance and EEG signals, feature selection is used for dimensionality reduction of EEG signals, feature fusion fuses different features into a feature vector, and decision fusion is used to fuse the decisions of different tests to obtain the final diagnosis result.
[0012] In the mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, the attention test stimulates the attention and reaction speed of the subject. The system sets up a 4×4 square array in an immersive virtual environment, randomly lights up a square, and the subject needs to quickly click the lit square using a controller of the corresponding color to collect the subject's cognitive performance, which is composed of score, reaction time, and total number of clicks; each time the subject responds correctly, the score increases by 1, and no points are deducted for non-response and wrong response, and the reaction time is the time interval between the click moment and the light-up moment;
[0013] The depth perception test stimulated the subject's spatial positioning ability and execution ability. The system set up multiple white balls in an immersive virtual environment. The subject had to click on the balls in order from far to near or from near to far. This process collected the subject's cognitive performance, which consisted of click sequence and reaction time. The reaction time was the time interval between clicking on the balls.
[0014] The working memory test stimulated the subject's spatial memory, working memory and executive ability. The test was divided into two stages: the memory stage and the execution stage. In the memory stage, the system set up a cabinet and multiple items in the cabinet in an immersive virtual reality environment. In the execution stage, the cabinet and items were set up in the immersive virtual reality environment. The subject needed to remember the placement of the items in the memory stage and choose the placement of the items in the execution stage. This process collected the subject's cognitive performance, which consisted of scores and reaction time. For each correct response of the subject, the score increased by 1, and no points were deducted for non-response and incorrect response. The reaction time was the time interval between the moment the subject clicked the placement location and the moment the item appeared.
[0015] During the test, the subjects' EEG data were collected to characterize the subjects' brain states during the test; and the start and end times were recorded in each test to segment the EEG signals of the corresponding tests in subsequent processing.
[0016] The described mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram (EEG) has a data preprocessing module for identifying and removing possible abnormal values and erroneous data (generally outliers) in cognitive performance data to ensure data reliability and accuracy; abnormal value detection is mainly based on reaction time in cognitive performance, and the distribution of reaction time in cognitive performance is approximately normal distribution, using the 3δ criterion, that is, it is considered that more than 99% of the data are concentrated within the range of 3 standard deviations above and below the mean; reaction time exceeding this range is an abnormal value, and the data record is deleted; the test start time and end time recorded in the cognitive performance are used to segment the EEG record of the corresponding test.
[0017] In the mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, the electroencephalogram signal processing and denoising module performs the following steps:
[0018] Positioning channel data: Loading channel location information that matches the recorded data; According to the national standard 10-20 for the placement of EEG electrodes, the wearable EEG device used in the system has two channels located in the frontal lobe (AF7, AF8) and two channels located in the temporal lobe (TP9, TP10);
[0019] Filtering: Use a Gaussian bandpass filter to extract signals in the frequency range of 0.5Hz-45Hz, retaining important neural electrical activities and excluding unnecessary noise;
[0020] Physiological artifact removal: Interference caused by eye movement is very common in EEG data. Independent component analysis (ICA) is used to remove physiological artifacts such as eye movement. The specific process is as follows:
[0021] The EEG data obtained has 4 electrodes distributed in 4 spatial positions, represented by x i (t), the information obtained by each electrode comes from 4 signal sources x j (t), the final EEG data can be expressed as:
[0022]
[0023] Among them, a ij Indicates the intensity of different signal sources captured at a certain electrode; it can be expressed using the vector matrix method as follows:
[0024] x=As
[0025] At this time, it is necessary to use the observable signal sequence x to estimate the mixing matrix A and the signal source s, which can be obtained using a common tool library to obtain the components of each signal source. The components with the attributes of physiological artifacts such as electrooculographic artifacts are deleted and the remaining data are reconstructed to obtain relatively pure EEG data.
[0026] Removal of non-physiological artifacts: It is inevitable to introduce additional signal interference into the original EEG signal. Since the EEG signal is a non-stationary signal, wavelet packet transformation is used to remove non-physiological artifacts in the original EEG signal. The specific process is as follows:
[0027] The dB4 wavelet in the dB wavelet is selected as the wavelet basis, and a 6-layer wavelet packet decomposition is performed to achieve an accuracy of 4Hz in the time domain; the wavelet packet coefficients obtained by the wavelet packet change are processed, and the soft threshold function is selected for threshold processing to achieve denoising. The soft threshold function expression is as follows:
[0028]
[0029] Where sgn(ω) is the sign function. The threshold λ is calculated by the following expression:
[0030]
[0031] Where σ is the estimated value of the noise standard deviation, and N is the length of the signal, that is, the number of samples;
[0032] EEG signal decomposition: The wavelet packet coefficients obtained by the above denoising are reconstructed, which is the inverse process of wavelet packet transformation, to obtain four typical frequency bands (waves). The first component 0-4Hz is mainly δ wave, the second component 4-8Hz is mainly θ wave, the third component 8-12Hz is mainly α wave, and the fourth to eighth components 12-32Hz are mainly β waves.
[0033] In the mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, cognitive performance feature extraction includes:
[0034] Task scores: Attention and working memory tests directly reflect the subject's ability to complete specific cognitive tasks;
[0035] Reaction time: refers to the average time required for subjects to respond to task stimuli;
[0036] Correctness: The proportion of correct responses of the subjects in the attention test;
[0037] Correlation score: In the depth perception task, the Kendall tau coefficient is used to calculate the correlation score between the response sequence and the correct sequence to evaluate the subject's spatial perception and execution ability. The calculation formula is:
[0038]
[0039] Where P represents the number of consistent pairwise permutations, Q represents the number of inconsistent pairwise permutations, and n is the total number of elements in the sequence;
[0040] The EEG signal feature extraction is as follows:
[0041] Average power: reflects the intensity of EEG activity in a specific frequency band. For each frequency band of different channels, the power spectral density (PSD) is calculated, and then the average power of the frequency band is calculated;
[0042] PSD is calculated using the following formula, which gives an expression of power per hertz:
[0043]
[0044] Among them, |X(f) 2 represents the simulation of the FFT result at frequency f, N is the total number of samples of the signal, and T is the sampling interval. X(f) is obtained by discrete Fourier transform:
[0045]
[0046] Where x(nT) is the signal value at time nT, k is the frequency index, and X(k) is the complex Fourier coefficient at frequency k / N;
[0047] Then the average power Pmean The calculation formula is:
[0048]
[0049] Spectral entropy: It measures the complexity of EEG signals and can be obtained by analyzing the distribution uniformity of the power spectrum within a specific frequency band. The calculation formula is:
[0050] FuzzyEn(m,n,r)=lnΦ m (n,r)-lnΦ m+1 (n,r)
[0051] Among them, Φ m (n,r), the calculation formula is:
[0052]
[0053] The sequence is defined as Where x0(i) is the mean of m consecutive x(i), let is the maximum value of the difference between the corresponding breakpoints of x(i) and x(j), and the fuzzy membership is used to measure their similarity:
[0054] Asymmetry index: used to quantify the functional asymmetry between the cerebral hemispheres. DASM refers to the difference in the average power of two symmetrical brain regions in a specific frequency band, while RASM is the ratio of the power of the two regions. The calculation formulas are:
[0055] DASM=P left -P right
[0056]
[0057] Among them, P left and P right Represents the average power of left and right brain regions in a specific frequency band, respectively.
[0058] The described mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram (EEG), feature selection of EEG signal features, the purpose of feature selection is to reduce the dimension of EEG signal features and improve the operation speed of the model; the average power of the β band located at AF8 and TP10, the spectral entropy located at TP10, and the DASM and RASM located at AF7-AF8 are selected as the optimal feature subsets of EEG signals, and the β band features show the best classification accuracy under the system.
[0059] In the mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, the feature fusion and decision fusion include:
[0060] Feature fusion: Using complementary information from different data sources, the optimal feature subset extracted from the EEG signal is initially integrated with the cognitive performance characteristics of each task;
[0061] Decision fusion integrates the prediction results of different classifiers through the following weighted voting formula:
[0062]
[0063] Among them, Vote c is the total vote score for a given category, Vote i,c is the vote for a given category in the i-th cognitive task, w i is the weight of the i-th cognitive task calculated based on the performance score; specifically, the weight of the classification w i Determined by the following formula:
[0064]
[0065] Among them, Performance i is the classification performance of the i-th cognitive task feature;
[0066] Finally, the final diagnosis result is obtained based on the total voting score.
[0067] The present invention constructs a mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, which is used to realize rapid diagnosis and screening of mild cognitive impairment. By combining virtual reality and electroencephalogram, the invention aims to address the problems of expensive, invasive and time-consuming clinical testing of mild cognitive impairment. It uses virtual reality to effectively stimulate the corresponding cognitive field of the subject, and uses electroencephalogram to reflect the brain state of the subject, thus realizing effective diagnosis of mild cognitive impairment. BRIEF DESCRIPTION OF THE DRAWINGS
[0068] Figure 1 It is an overall flow chart of the mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram of the present invention;
[0069] Figure 2 It is a specific implementation scenario of the present invention;
[0070] Figure 3 It is the virtual reality test of the present invention; a attention test, b depth perception test, c working memory test;
[0071] Figure 4 It is a schematic diagram of the wavelet packet change decomposition tree of the present invention;
[0072] Figure 5 This is a schematic diagram of the data processing module results of the present invention;
[0073] Figure 6A comparison chart of cognitive performance of different groups of people in the diagnosis system of mild cognitive impairment based on virtual reality and electroencephalogram;
[0074] Figure 7 A comparative diagram of brain activities during cognitive processes among different groups of people in the diagnosis system for mild cognitive impairment based on virtual reality and electroencephalogram; DETAILED DESCRIPTION
[0075] The present invention is described in detail below in conjunction with specific embodiments.
[0076] A mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, including a virtual reality test module, a data preprocessing and denoising module, and a decision fusion module based on voting strategy. The overall flow chart is as follows: Figure 1 As shown, virtual reality testing, such as Figure 3 As shown, including the following:
[0077] A1 uses virtual reality equipment to deploy virtual reality tests such as attention test, depth perception test and working memory test. The subjects need to wear virtual reality equipment and wearable EEG equipment at the same time, such as Figure 2 As shown. In this process, the virtual reality test collects the cognitive performance of the subjects during the test and simultaneously records the start and end times of each test. The wearable EEG device collects EEG signals;
[0078] Attention tests stimulated the subject's attention and reaction speed, such as Figure 2 As shown in a, the system sets up a 4×4 square array in an immersive virtual environment, randomly lights up a square (displayed as blue or yellow), and the subject needs to quickly click the lit square using the controller of the corresponding color. In this process, the cognitive performance of the subject is collected, and the cognitive performance consists of score, reaction time, and total number of clicks. For each correct response of the subject, the score increases by 1, and no points are deducted for non-response and incorrect response. The reaction time is the time interval between the click time and the light-up time;
[0079] The depth perception test stimulated the subject's spatial positioning ability and execution ability. The system set up multiple white balls in an immersive virtual environment, such as Figure 2 As shown in b, the subjects need to click on the balls in order from far to near or from near to far. This process collects the subjects' cognitive performance, which consists of click sequence and reaction time. Reaction time is the time interval between clicking on the balls;
[0080] The working memory test stimulated the subjects' spatial memory, working memory, and executive function, e.g. Figure 2As shown in Figure c, the test is divided into two stages, the memory stage and the execution stage. In the memory stage, the system sets up cabinets and multiple items in the cabinets in an immersive virtual reality environment. In the execution stage, cabinets and items are set up in an immersive virtual reality environment. The subjects need to memorize the placement of the items in the memory stage, and choose the placement of the items in the execution stage. This process collects the cognitive performance of the subjects, which consists of scores and reaction time. For each correct response of the subject, the score increases by 1, and no points are deducted for non-response and incorrect response. The reaction time is the time interval between the moment the subject clicks the placement location and the moment the object appears;
[0081] During the test, the EEG data of the subjects were collected to characterize the brain state of the subjects during the test. The start and end times of each test were recorded to segment the EEG signals of the corresponding test in subsequent processing.
[0082] A2 performs data preprocessing on the collected cognitive performance data to identify and remove possible abnormal values and erroneous data (generally outliers) in the cognitive performance data to ensure the reliability and accuracy of the data. Outlier detection is mainly based on the reaction time in cognitive performance. The distribution of reaction time in cognitive performance is approximately normal distribution. The 3δ criterion is used, which means that more than 99% of the data are concentrated within 3 standard deviations above and below the mean. If the reaction time exceeds this range, it is an abnormal value and the data record is deleted. Use the test start and end times recorded in the cognitive performance to segment the EEG records of the corresponding test.
[0083] A3 processes and denoises the collected EEG data to obtain cleaner EEG data. It mainly includes positioning channel data, Gaussian bandpass filtering, physiological artifact removal, non-physiological artifact removal and decomposition. The specific steps are as follows:
[0084] A31. Positioning channel data: Load the channel location information that matches the recorded data. According to the national standard 10-20 for the placement of EEG electrodes, the wearable EEG device used in the system has two channels located in the frontal lobe (AF7, AF8) and two channels located in the temporal lobe (TP9, TP10);
[0085] A32. Filtering: Use a Gaussian bandpass filter to extract signals in the frequency range of 0.5Hz-45Hz, retaining important neural electrical activities and excluding unnecessary noise, such as ultra-low-frequency and high-frequency myoelectric interference;
[0086] A33. Physiological artifact removal: Interference caused by eye movement is very common in EEG data. Independent component analysis (ICA) is used to remove physiological artifacts such as eye movement. The specific process is as follows:
[0087] The EEG data obtained has 4 electrodes distributed in 4 spatial positions, represented by x i (t), the information obtained by each electrode comes from 4 signal sources x j (t), the final EEG data can be expressed as:
[0088]
[0089] Among them, a ij Indicates the intensity of different signal sources captured at a certain electrode. It can be expressed using the vector matrix method as follows:
[0090] x=As
[0091] At this time, it is necessary to use the observable signal sequence x to estimate the mixing matrix A and the signal source s, which can be obtained using a common tool library to obtain the components of each signal source. The components with the attributes of physiological artifacts such as electrooculographic artifacts are deleted and the remaining data are reconstructed to obtain relatively pure EEG data.
[0092] A34. Non-physiological artifact removal: It is inevitable to introduce additional signal interference into the original EEG signal, such as external environment, equipment operation, etc. Since the EEG signal is a non-stationary signal, wavelet packet transformation is used to remove non-physiological artifacts in the original EEG signal. The specific process is as follows:
[0093] The dB4 wavelet in the dB wavelet is selected as the wavelet basis, and a 6-layer wavelet packet decomposition is performed (such as Figure 4 As shown), the accuracy of 4Hz is achieved in the time domain. The wavelet packet coefficients obtained by wavelet packet change are processed, and the soft threshold function is used for threshold processing to achieve denoising. The soft threshold function expression is as follows:
[0094]
[0095] Where sgn(ω) is the sign function. The threshold λ is calculated by the following expression:
[0096]
[0097] Where σ is an estimate of the standard deviation of the noise, and N is the length of the signal (i.e., the number of samples);
[0098] A35. EEG signal decomposition: The wavelet packet coefficients obtained by denoising are reconstructed (the inverse process of wavelet packet transformation) to obtain four typical frequency bands (waves). The first component (0-4Hz) is mainly δ waves, the second component (4-8Hz) is mainly θ waves, the third component (8-12Hz) is mainly α waves, and the fourth to eighth components (12-32Hz) are mainly β waves.
[0099] EEG signal processing and denoising results are shown in Figure 2. Figure 5shown.
[0100] A4 is a decision fusion based on voting strategy. It extracts features from the processed cognitive performance data and EEG waves of different frequency bands in different channels obtained in the above steps, performs feature selection on EEG features to reduce the dimension, fuses cognitive performance features and EEG features, and finally performs decision fusion. The specific steps are as follows:
[0101] A41. Cognitive performance features are extracted as follows:
[0102] Task scores: Attention and working memory tests directly reflect the subject's ability to complete specific cognitive tasks;
[0103] Reaction time: refers to the average time required for subjects to respond to task stimuli;
[0104] Correctness: The proportion of correct responses of the subjects in the attention test;
[0105] Correlation score: In the depth perception task, the Kendall tau coefficient is used to calculate the correlation score between the response sequence and the correct sequence to evaluate the subject's spatial perception and execution ability. The calculation formula is:
[0106]
[0107] Where P represents the number of consistent pairwise permutations, Q represents the number of inconsistent pairwise permutations, and n is the total number of elements in the sequence;
[0108] Figure 6 This is a comparison of cognitive performance characteristics of different populations; it can be seen intuitively that the mean score of the mild cognitive impairment (MCI) group is lower than that of the healthy control (HC) group, and the standard deviation is greater than that of the HC group; the mean response time of the mild cognitive impairment group is slower than that of the HC group, and the standard deviation is also greater than that of the HC group. This shows that the cognitive performance of people with mild cognitive impairment is generally lower than that of the HC group, and there are large differences within the population. The scores and response times conform to the normal distribution, and there are significant differences between the two groups, which shows the effectiveness of virtual reality testing in distinguishing the two groups.
[0109] A42. The EEG signal features are extracted as follows:
[0110] Average power: reflects the intensity of EEG activity in a specific frequency band. For each frequency band of different channels, the power spectral density (PSD) is calculated, and then the average power of the frequency band is calculated;
[0111] PSD is calculated using the following formula, which gives an expression of power per hertz:
[0112]
[0113] Among them, |X(f) 2 represents the simulation of the FFT result at frequency f, N is the total number of samples of the signal, and T is the sampling interval. X(f) is obtained by discrete Fourier transform:
[0114]
[0115] Where x(nT) is the signal value at time nT, k is the frequency index, and X(k) is the complex Fourier coefficient at frequency k / N;
[0116] Then the average power P mean The calculation formula is:
[0117]
[0118] Spectral entropy: It measures the complexity of EEG signals and can be obtained by analyzing the distribution uniformity of the power spectrum within a specific frequency band. The calculation formula is:
[0119] FuzzyEn(m,n,r)=lnΦ m (n,r)-lnΦ m+1 (n,r)
[0120] Among them, Φ m (n,r), the calculation formula is:
[0121]
[0122] The sequence is defined as Where x0(i) is the mean of m consecutive x(i), let is the maximum value of the difference between the corresponding breakpoints of x(i) and x(j), and the fuzzy membership is used to measure their similarity:
[0123] Asymmetry index: used to quantify the functional asymmetry between the cerebral hemispheres. DASM refers to the difference in the average power of two symmetrical brain regions in a specific frequency band, while RASM is the ratio of the power of the two regions. The calculation formulas are:
[0124] DASM=P left -P right
[0125]
[0126] Among them, P left and P right They represent the average power of the left and right brain regions in a specific frequency band, respectively;
[0127] Figure 7Comparison of EEG activity between different groups of people; This figure presents a visual representation of the differences in brain activity between people with mild cognitive impairment (MCI) and HC in different EEG frequency bands—δ, θ, α, and β. It can be seen that in the β frequency band, the activity between the two groups showed significant differences, which may be due to the β band being related to the ability to regulate attention and engage in cognitive tasks that require rapid response, which shows that virtual reality testing can cause EEG differences between different groups of people;
[0128] A43. Feature selection of EEG signal features. The purpose of feature selection is to reduce the dimension of EEG signal features and improve the operation speed of the model. The average power of the β band located at AF8 and TP10, the spectral entropy located at TP10, and the DASM and RASM located at AF7-AF8 are selected as the optimal feature subset. The β band features show the best classification accuracy under the system.
[0129] A44. Feature fusion and decision fusion mechanism based on voting strategy;
[0130] Feature fusion: Using the complementary information from different data sources, the optimal feature subset extracted from the EEG signal is initially integrated with the cognitive performance characteristics of each task.
[0131] Decision fusion integrates the prediction results of different classifiers through the following weighted voting formula:
[0132]
[0133] Among them, Vote c is the total vote score for a given category, Vote i,c is the vote for a given category in the i-th cognitive task, w i is the weight of the i-th cognitive task calculated based on the performance score. Specifically, the weight of the classification w i Determined by the following formula:
[0134]
[0135] Among them, Performance i is the classification performance of the i-th cognitive task feature;
[0136] Finally, the final diagnosis result is obtained according to the total voting score, as shown in Table 1 below, which illustrates the final classification effect from the perspective of accuracy, precision, recall, F1 score and Matthews coefficient.
[0137] Table 1 Classification effect of mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram
[0138]
[0139] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.
Claims
1. A mild cognitive impairment diagnosis system based on virtual reality and electroencephalogram, characterized in that: Including virtual reality testing module, data preprocessing module, and decision fusion module based on voting strategy: Virtual reality test module: This module contains targeted virtual reality tests, including attention test, depth perception test and working memory test. It uses immersive virtual reality technology to stimulate the subject's attention, depth perception ability and working memory cognitive fields respectively, and collects the subject's cognitive performance and EEG data in the process, so as to diagnose the subject's mild cognitive impairment from the two dimensions of cognitive performance and brain state; at the same time, the virtual reality test collects the start and end time of each test, so as to segment the EEG data of each test in the subsequent processing; Data preprocessing module: After the test, the cognitive performance and EEG data of the virtual reality test are preprocessed to identify and remove possible outliers and erroneous data to ensure the reliability and accuracy of the data; The decision fusion module based on voting strategy includes feature extraction, feature selection, feature fusion and decision fusion; feature extraction is used to extract features with strong representation ability of cognitive performance and EEG signals, feature selection is used for dimensionality reduction of EEG signals, feature fusion fuses different features into a feature vector, and decision fusion is used to fuse the decisions of different tests to obtain the final diagnosis result.
2. A virtual reality and electroencephalogram-based mild cognitive impairment diagnostic system according to claim 1, characterized in that: The attention test stimulated the subject's attention and reaction speed. The system set up a 4×4 square array in an immersive virtual environment, randomly lit up a square, and the subject had to quickly use the controller of the corresponding color to click on the lit square to collect the subject's cognitive performance, which consisted of score, reaction time, and total number of clicks. For each correct response of the subject, the score increased by 1, and no points were deducted for non-response and incorrect response. The reaction time was the time interval between the click time and the light-up time. The depth perception test stimulated the subject's spatial positioning ability and execution ability. The system set up multiple white balls in an immersive virtual environment, and the subject had to click on the balls in order from far to near or from near to far. This process collected the subject's cognitive performance, which consisted of the click sequence and reaction time. The reaction time was the time interval between clicking the balls. The working memory test stimulated the subject's spatial memory, working memory and executive ability. The test was divided into two stages: the memory stage and the execution stage. In the memory stage, the system set up a cabinet and multiple objects in the cabinet in an immersive virtual reality environment. In the execution stage, the cabinet and objects were set up in the immersive virtual reality environment. The subject needed to remember the placement of the objects in the memory stage and choose the placement of the objects in the execution stage. This process collected the subject's cognitive performance, which consisted of scores and reaction time. For each correct response of the subject, the score increased by 1, and no points were deducted for non-response and incorrect response. The reaction time was the time interval between the moment the subject clicked the placement location and the moment the object appeared. During the test, the subjects’ EEG data were collected to characterize the subjects’ brain states during the test; The start and end times of each test were recorded to segment the EEG signals of the corresponding test in subsequent processing.
3. A virtual reality and electroencephalogram-based mild cognitive impairment diagnostic system according to claim 1, characterized in that: The data preprocessing module is used to identify and remove possible outliers and erroneous data in cognitive performance data to ensure data reliability and accuracy. Outlier detection is mainly based on the reaction time in cognitive performance. The distribution of reaction time in cognitive performance is approximately normal. The 3δ criterion is used, which means that more than 99% of the data are concentrated within 3 standard deviations above and below the mean. If the reaction time exceeds this range, it is an outlier and the data record is deleted. The test start and end times recorded in the cognitive performance are used to segment the EEG records of the corresponding test.
4. A virtual reality and electroencephalogram-based mild cognitive impairment diagnostic system according to claim 1, characterized in that: The EEG signal processing and denoising module performs the following steps: Positioning channel data: Loading channel location information that matches the recorded data; According to the national standard 10-20 for the placement of EEG electrodes, the wearable EEG device used in the system has two channels located in the frontal lobe (AF7, AF8) and two channels located in the temporal lobe (TP9, TP10); Filtering: Use a Gaussian bandpass filter to extract signals in the frequency range of 0.5Hz-45Hz, retaining important neural electrical activities and excluding unnecessary noise; Physiological artifact removal: Interference caused by eye movement is very common in EEG data. Independent component analysis (ICA) is used to remove physiological artifacts such as eye movement. The specific process is as follows: The EEG data obtained has 4 electrodes distributed in 4 spatial positions, represented by x i (t), the information obtained by each electrode comes from 4 signal sources x j (t), the final EEG data can be expressed as: Among them, a ij Indicates the intensity of different signal sources captured at a certain electrode; it can be expressed using the vector matrix method as follows: x=As At this time, it is necessary to use the observable signal sequence x to estimate the mixing matrix A and the signal source s, which can be obtained using a common tool library to obtain the components of each signal source. The components with the attributes of physiological artifacts such as electrooculographic artifacts are deleted and the remaining data are reconstructed to obtain relatively pure EEG data. Removal of non-physiological artifacts: It is inevitable to introduce additional signal interference into the original EEG signal. Since the EEG signal is a non-stationary signal, wavelet packet transformation is used to remove non-physiological artifacts in the original EEG signal. The specific process is as follows: The dB4 wavelet in the dB wavelet is selected as the wavelet basis, and a 6-layer wavelet packet decomposition is performed to achieve an accuracy of 4Hz in the time domain; the wavelet packet coefficients obtained by the wavelet packet change are processed, and the soft threshold function is selected for threshold processing to achieve denoising. The soft threshold function expression is as follows: Where sgn(ω) is the sign function; the threshold λ is calculated by the following expression: Where σ is the estimated value of the noise standard deviation, and N is the length of the signal, that is, the number of samples; EEG signal decomposition: The wavelet packet coefficients obtained by the above denoising are reconstructed, which is the inverse process of wavelet packet transformation, to obtain four typical frequency bands (waves). The first component 0-4Hz is mainly δ wave, the second component 4-8Hz is mainly θ wave, the third component 8-12Hz is mainly α wave, and the fourth to eighth components 12-32Hz are mainly β waves.
5. A virtual reality and electroencephalogram-based mild cognitive impairment diagnosis system according to claim 1, characterized in that: Cognitive performance feature extraction includes: Task scores: Attention and working memory tests directly reflect the subject's ability to complete specific cognitive tasks; Reaction time: refers to the average time required for subjects to respond to task stimuli; Correctness: The proportion of correct responses of the subjects in the attention test; Correlation score: In the depth perception task, the Kendall tau coefficient is used to calculate the correlation score between the response sequence and the correct sequence to evaluate the subject's spatial perception and execution ability. The calculation formula is: Where P represents the number of consistent pairwise permutations, Q represents the number of inconsistent pairwise permutations, and n is the total number of elements in the sequence; The EEG signal feature extraction is as follows: Average power: reflects the intensity of EEG activity in a specific frequency band; for each frequency band of different channels, the power spectral density (PSD) is calculated, and then the average power of the frequency band is calculated; PSD is calculated using the following formula, which gives an expression of power per hertz: Among them, |X(f) 2 represents the simulation of the FFT result at frequency f, N is the total number of samples of the signal, T is the sampling interval; X(f) is obtained by discrete Fourier transform: Where x(nT) is the signal value at time nT, k is the frequency index, and X(k) is the complex Fourier coefficient at frequency k / N; Then the average power P mean The calculation formula is: Spectral entropy: It measures the complexity of EEG signals and can be obtained by analyzing the distribution uniformity of the power spectrum within a specific frequency band. The calculation formula is: FuzzyEn(m,n,r)=lnΦ m (n,r)-lnΦ m+1 (n,r) Among them, Φ m (n,r), the calculation formula is: The sequence is defined as Where x0(i) is the mean of m consecutive x(i), let is the maximum value of the difference between the corresponding breakpoints of x(i) and x(j), and the fuzzy membership is used to measure their similarity: Asymmetry index: used to quantify the functional asymmetry between the cerebral hemispheres; DASM refers to the difference in the average power of two symmetrical brain regions in a specific frequency band, while RASM is the ratio of the power of the two regions; the calculation formulas are: DASM=P left -P right Among them, P left and P right Represents the average power of left and right brain regions in a specific frequency band, respectively.
6. A virtual reality and electroencephalogram-based mild cognitive impairment diagnostic system according to claim 1, characterized in that: Feature selection of EEG signal features. The purpose of feature selection is to reduce the dimension of EEG signal features and improve the calculation speed of the model. The average power of the β band located at AF8 and TP10, the spectral entropy located at TP10, and DASM and RASM located at AF7-AF8 are selected as the optimal feature subsets of EEG signals. The β band features show the best classification accuracy under the system.
7. A virtual reality and electroencephalogram-based mild cognitive impairment diagnosis system according to claim 1, characterized in that: The feature fusion and decision fusion include: Feature fusion: Using complementary information from different data sources, the optimal feature subset extracted from the EEG signal is initially integrated with the cognitive performance characteristics of each task; Decision fusion integrates the prediction results of different classifiers through the following weighted voting formula: Among them, Vote c is the total vote score for a given category, Vote i,c is the vote for a given category in the i-th cognitive task, w i is the weight of the i-th cognitive task calculated based on the performance score; specifically, the weight of the classification w i Determined by the following formula: Among them, Performance i is the classification performance of the i-th cognitive task feature; Finally, the final diagnosis result is obtained based on the total voting score.
Citation Information
Cited By
Psychological nursing management method and system based on electroencephalogram biofeedback
CN121400835A
