Cognitive impairment assessment system based on virtual reality and electroencephalogram signals
The VR-based EEG system addresses subjective assessment issues by using VR and EEG data processing to enhance accuracy and efficiency in cognitive dysfunction evaluation.
Patent Information
- Application Number
- CN202411098733.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-12
- Publication Date
- 2024-12-06
AI Technical Summary
Existing cognitive dysfunction assessment methods rely heavily on subjective assessor judgment, leading to inaccurate results and high labor costs, with low evaluation efficiency.
A system utilizing VR glasses and a head-mounted EEG acquisition device to collect and process EEG signals from multiple brain regions, combined with eye and head movement data, to generate accurate cognitive dysfunction levels through a deep learning model.
The system provides objective and efficient cognitive dysfunction assessments with reduced subjectivity, improving accuracy and reducing labor costs.
Smart Images

Figure CN119089169A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of virtual reality and electroencephalogram signals, and in particular to a cognitive dysfunction assessment system based on virtual reality and electroencephalogram signals. Background Art
[0002] Cognitive dysfunction is a neurological syndrome with cognitive impairment such as memory and orientation as the main symptoms, which gradually progresses from mild cognitive impairment to dementia. In the prior art, the method of evaluating the cognitive dysfunction of the assessor is mostly: the assessor fills in some cognitive impairment scales under the guidance of the staff, such as the Wechsler Intelligence Scale (WAIS), the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment Battery (MOCAB), etc., and the assessor's cognitive dysfunction is determined by the assessor's filling of the scale. This cognitive dysfunction method incorporates the assessor's subjective judgment factor, which will greatly affect the accuracy of the final evaluation result. It is also affected by the assessor's education level and knowledge reserve, which leads to inaccurate final evaluation results. In addition, this cognitive dysfunction method has low evaluation efficiency and high labor cost. Summary of the invention
[0003] In view of the problems and shortcomings of the prior art, the present invention provides a cognitive dysfunction assessment system based on virtual reality and electroencephalogram signals.
[0004] The present invention solves the above technical problems through the following technical solutions:
[0005] The present invention provides a cognitive dysfunction assessment system based on virtual reality and electroencephalogram signals, which is characterized in that it includes VR glasses and a head-mounted electroencephalogram acquisition device for an assessor to wear, and a server that interacts with the VR glasses and the head-mounted electroencephalogram acquisition device, wherein the server includes a control module and a display module;
[0006] The control module is used to control the VR glasses to play and display A virtual reality scene videos in sequence with rest intervals for the evaluator to watch, the virtual reality scene videos are videos with sound, the viewing difficulty of the A virtual reality scene videos ranges from none to one, from low to high, and the viewing time of each virtual reality scene video is the same, and in the process of the evaluator watching each virtual reality scene video, the head-mounted electroencephalogram acquisition device is controlled to continuously collect electroencephalogram signals of multiple channels in the frontal lobe brain area, the parietal lobe brain area, the occipital lobe brain area and the temporal lobe brain area at a set sampling frequency, A is a positive integer and A≥2;
[0007] The control module is also used to perform denoising filtering on the EEG signals of each channel of each brain region corresponding to each virtual reality scene video to obtain the corresponding pre-processed EEG signals;
[0008] The control module is also used for the preprocessed EEG signals of each channel of each brain region corresponding to the a-th virtual reality scene video: judging whether there are abnormal signals at each sampling point in the preprocessed EEG signals of each channel of each brain region, and taking the preprocessed EEG signals of each channel of each brain region as qualified EEG signals when there are no abnormal signals, and performing abnormal signal processing when there are abnormal signals and taking the EEG signals of each channel of each brain region obtained after the abnormal signal processing as qualified EEG signals, respectively analyzing the variance of the qualified EEG signals of each channel of each brain region, respectively performing Fourier transform on the qualified EEG signals of each channel of each brain region, and analyzing the power spectral density of the δ wave, θ wave, α wave, β wave and γ wave frequency bands of each channel, analyzing the connectivity coefficients between each brain region, and constructing a one-dimensional feature matrix (E f 、E δ 、E θ 、E α 、E β 、E γ , D f , D δ , D θ , D α , D β , D γ , Z f , Z δ , Z θ , Z α , Z β , Z γ 、N f 、N δ 、N θ 、N α 、N β 、N γ , L ED , L EZ , L EN , L DZ , L DN , L ZN ), a is taken as 1-A in sequence, so as to obtain an A-dimensional feature matrix consisting of feature matrices corresponding to A virtual reality scene videos;
[0009] Among them, E f The average value of the variance of qualified EEG signals of each channel in the frontal lobe brain area, E δ The average value of the delta wave power spectral density of qualified EEG signals of each channel in the frontal lobe brain area, D f The average value of the variance of qualified EEG signals of each channel in the parietal brain area, D δ The average value of the delta wave power spectral density of the qualified EEG signals of each channel in the parietal brain area, Z f Represents the average value of the variance of qualified EEG signals of each channel in the occipital lobe brain area, Z δThe average value of the delta wave power spectral density of the qualified EEG signals of each channel in the occipital lobe brain area, N f Represents the average value of the variance of qualified EEG signals of each channel in the temporal lobe brain area, N δ The average value of the delta wave power spectral density of qualified EEG signals of each channel in the temporal lobe brain area, L ED represents the connectivity coefficient between the frontal lobe and the parietal lobe, L EZ represents the connectivity coefficient between the frontal lobe and the occipital lobe, L EN represents the connectivity coefficient between the frontal lobe and the temporal lobe, L DZ represents the connectivity coefficient between the parietal lobe and the occipital lobe, L DN represents the connectivity coefficient between the parietal lobe and the temporal lobe, L ZN Represents the connectivity coefficient between the occipital lobe and temporal lobe brain regions;
[0010] The control module is also used to input the A-dimensional feature matrix into the trained cognitive dysfunction assessment model for prediction, so as to output the evaluator's cognitive dysfunction level and control the display module to display, and the cognitive dysfunction level includes no cognitive dysfunction level, mild cognitive dysfunction level, moderate cognitive dysfunction level and severe cognitive dysfunction level.
[0011] The positive and progressive effects of the present invention are:
[0012] In the present invention, abnormal signal processing is performed when abnormal conditions exist in the EEG signal, so that a qualified EEG signal can be obtained. By using the qualified EEG signal for feature extraction, more accurate feature extraction values can be obtained, making the final cognitive dysfunction assessment result more accurate.
[0013] In the present invention, features extracted from EEG signals, features extracted from eye movement information, and features extracted from head movement information are input into a cognitive dysfunction assessment model for prediction, so as to output a more accurate cognitive dysfunction level of the assessor.
[0014] The present invention has a high degree of intelligence, high evaluation efficiency, and low labor cost, and greatly reduces the influence of the subjective judgment factors of the evaluator. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 Schematic diagram of the structure of a cognitive dysfunction assessment system according to a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0016] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0017] like Figure 1 As shown, this embodiment provides a cognitive dysfunction assessment system based on virtual reality and EEG signals, which includes VR glasses 10 and a head-mounted EEG acquisition device 20 for the assessor to wear, and a server 30 that interacts with the VR glasses 10 and the head-mounted EEG acquisition device 20. The VR glasses 10 are provided with a left eye tracking camera 11, a right eye tracking camera 12 and a posture sensor 13, and the server 30 includes a control module 31 and a display module 32.
[0018] The control module 31 is used to control the VR glasses 10 to play and display A virtual reality scene videos in sequence with rest intervals for the evaluator to watch. The virtual reality scene video is a video with sound. The viewing difficulty of the A virtual reality scene videos increases from none to yes and from low to high. The viewing time of each virtual reality scene video is the same. In the process of the evaluator watching each virtual reality scene video, the head-mounted EEG acquisition device 20 is controlled to continuously collect EEG signals of multiple channels in the frontal lobe brain area, parietal lobe brain area, occipital lobe brain area and temporal lobe brain area at a set sampling frequency. A is a positive integer and A≥2. In the process of the evaluator watching each virtual reality scene video, the left eye movement tracking camera 11 is controlled to collect left eye movement video images, the right eye movement tracking camera 12 is controlled to collect right eye movement video images, and the posture sensor 13 is controlled to collect head movement conditions.
[0019] For example, in this embodiment, the VR glasses 10 are configured to play and display three virtual reality scene videos in sequence with rest intervals for the evaluator to watch. The viewing difficulty of the three virtual reality scene videos is no difficulty, slightly difficult, and difficult.
[0020] The viewing time of each virtual reality scene video is the same in order to collect the EEG signals of each channel in each brain region with the same time series length, so that each sampling point in each channel of each brain region can be matched one by one.
[0021] The control module 31 is also used to perform denoising filtering on the EEG signals of each channel of each brain region corresponding to each virtual reality scene video to obtain the corresponding pre-processed EEG signals.
[0022] The control module 31 is also used for the preprocessed EEG signals of each channel of each brain region corresponding to the a-th virtual reality scene video: first, the EEG signals of B sampling points from the head end and the EEG signals of C sampling points from the tail end of the preprocessed EEG signals of each channel of each brain region are intercepted to obtain the processed segment EEG signals of each channel of each brain region, and then it is determined whether there are abnormal signals at each sampling point in the processed segment EEG signals of each channel of each brain region. When there are no abnormal signals, there is no need to perform abnormal signal processing, and the processed segment EEG signals of each channel of each brain region are used as qualified EEG signals. When there are abnormal signals, abnormal signal processing is performed and the EEG signals of each channel of each brain region obtained after the abnormal signal processing are used as qualified EEG signals. B and C are positive integers, and a is 1-A in sequence, so as to obtain qualified EEG signals of each channel of each brain region corresponding to A virtual reality scene videos.
[0023] When there are abnormal signals, the abnormal signals are processed and the EEG signals of each channel in each brain region obtained after the abnormal signal processing are used as qualified EEG signals. The specific processing process is: first determine whether there is a situation in which the EEG signals of at least one channel in a certain brain region have multiple consecutive sampling points with abnormal signal amplitudes. If the answer is yes, directly delete the EEG signals of the channel in the brain region with multiple consecutive sampling points with abnormal signal amplitudes and then splice them. At the same time, delete the EEG signals of other channels in the brain region and the corresponding deleted segments of each channel in other brain regions and then splice them. Then determine whether there is a situation in which the EEG signals of at least one channel in a certain brain region have a single sampling point with abnormal signal amplitudes. If the answer is yes, further determine and remove the single amplitude abnormal sampling point in the channel in the brain region. After deleting the sampling point signal and deleting the EEG signals of other channels of the brain region and the corresponding deleted points of each channel of other brain regions, whether the time series length of the EEG signal of a certain channel after splicing reaches the set time series length, if it is, the EEG signal after deleting and splicing each single sampling point of each channel of each brain region is used as a qualified EEG signal, when the time series length of the spliced EEG signal does not reach the set time series length, the single sampling point signal of the channel of the brain region with abnormal amplitude is corrected, and the corrected EEG signal of each channel of each brain region is used as a qualified EEG signal, and when there is no abnormal amplitude of a single sampling point signal, the EEG signal after deleting and splicing each continuous multiple sampling points of each channel of each brain region is used as a qualified EEG signal.
[0024] When there are no consecutive abnormal signal amplitudes of multiple sampling points in the EEG signal of a certain channel in a certain brain region, it indicates that there is a single abnormal signal amplitude at a single sampling point in the EEG signal of a certain channel in at least one brain region. The EEG signal of the single abnormal amplitude sampling point in the channel of the brain region is directly deleted and then spliced. At the same time, the EEG signals of the corresponding deleted points of other channels of the brain region and each channel of other brain regions are deleted and then spliced. The EEG signals after deleting and splicing each single sampling point in each channel of each brain region are used as qualified EEG signals.
[0025] The signal of a single abnormal amplitude sampling point of the channel in the brain region is corrected by replacing the average amplitude of the previous sampling point and the next sampling point of the abnormal amplitude sampling point of the channel in the brain region with the new amplitude of the abnormal amplitude sampling point.
[0026] In this embodiment, it is first determined whether there is a situation where the amplitude of the signal of multiple sampling points is abnormal, and then it is determined whether there is a situation where the amplitude of the signal of a single sampling point is abnormal. In the case of abnormal amplitude of the signal of multiple sampling points, the EEG signals with abnormal amplitude of continuous multiple sampling points are directly deleted. After the EEG signals with abnormal amplitude of multiple sampling points are processed, it is determined whether there is a situation where the amplitude of the signal of a single sampling point is abnormal. In the case of abnormal amplitude of the signal of a single sampling point, there are two situations: 1. If the time series length after deleting each EEG signal with abnormal amplitude of a single sampling point meets the condition, the EEG signals with abnormal amplitude of each single sampling point are directly deleted to form a qualified EEG signal; 2. If the time series length after deleting each EEG signal with abnormal amplitude of a single sampling point does not meet the condition, the EEG signals with abnormal amplitude of each single sampling point are corrected to obtain a corrected qualified EEG signal. If there is no abnormal amplitude of multiple sampling points, it means that there must be abnormal amplitude of single sampling point signal. Since there is no deletion of continuous EEG signals, the evaluator will not make large movements during the viewing process, so there will not be many abnormal amplitudes of single sampling point signals. Therefore, the EEG signals with abnormal amplitudes of each single sampling point can be directly deleted to form a qualified EEG signal. After processing the amplitude abnormal signal, a qualified EEG signal can be obtained. Using the qualified EEG signal for feature extraction can obtain more accurate feature extraction values, making the final cognitive dysfunction assessment result more accurate.
[0027] The control module 31 is also used for: for the left eye movement video image and the right eye movement video image corresponding to the a-th virtual reality scene video, positioning the pupil center for each frame of the left eye movement video image, obtaining the change of the left pupil center coordinate in time sequence, analyzing the change values of the left pupil center coordinate twice before and after, and calculating the variance X of the change value of the left pupil center coordinate 左 , locate the pupil center of each frame of the right eye movement video image, obtain the change of the right pupil center coordinate in time sequence, analyze the change values of the right pupil center coordinate twice before and after, and calculate the variance X of the change value of the right pupil center coordinate 右 , a is 1-A respectively.
[0028] The control module 31 is also used for the head movement corresponding to the ath virtual reality scene video: performing head positioning for each collected head movement, obtaining the change of the head fixed point coordinates in time sequence, analyzing the change values of the head fixed point coordinates twice before and after, and calculating the variance X of the change value of the head fixed point coordinates.头 , a is 1-A respectively.
[0029] The control module 31 is also used for analyzing the variance of the qualified EEG signals of each channel of each brain region corresponding to the ath virtual reality scene video, performing Fourier transform on the qualified EEG signals of each channel of each brain region, analyzing the power spectrum density of the δ wave, θ wave, α wave, β wave and γ wave frequency band of each channel, analyzing the connectivity coefficients between each brain region, and constructing a one-dimensional feature matrix (E f 、E δ 、E θ 、E α 、E β 、E γ , D f , D δ , D θ , D α , D β , D γ , Z f , Z δ , Z θ , Z α , Z β , Z γ 、N f 、N δ 、N θ 、N α 、N β 、N γ , L ED , L EZ , L EN , L DZ , L DN , L ZN , X 左 , X 右 , X 头 ), a is taken as 1-A in sequence, so as to obtain an A-dimensional feature matrix composed of feature matrices corresponding to A virtual reality scene videos.
[0030] Among them, E f The average value of the variance of qualified EEG signals of each channel in the frontal lobe brain area, E δ The average value of the delta wave power spectral density of qualified EEG signals of each channel in the frontal lobe brain area, E θ The average value of the power spectrum density of the θ wave of qualified EEG signals of each channel in the frontal lobe brain area, E α The average value of the alpha wave power spectrum density of qualified EEG signals of each channel in the frontal lobe brain area, E β The average value of the β-wave power spectral density of qualified EEG signals of each channel in the frontal lobe brain area, E γThe average value of the gamma wave power spectrum density of qualified EEG signals of each channel in the frontal lobe brain area, D f The average value of the variance of qualified EEG signals of each channel in the parietal brain area, D δ The average value of the delta wave power spectral density of the qualified EEG signals of each channel in the parietal brain area, Z f Represents the average value of the variance of qualified EEG signals of each channel in the occipital lobe brain area, Z δ The average value of the delta wave power spectral density of the qualified EEG signals of each channel in the occipital lobe brain area, N f Represents the average value of the variance of qualified EEG signals of each channel in the temporal lobe brain area, N δ The average value of the delta wave power spectral density of qualified EEG signals of each channel in the temporal lobe brain area, L ED represents the connectivity coefficient between the frontal lobe and the parietal lobe, L EZ represents the connectivity coefficient between the frontal lobe and the occipital lobe, L EN represents the connectivity coefficient between the frontal lobe and the temporal lobe, L DZ represents the connectivity coefficient between the parietal lobe and the occipital lobe, L DN represents the connectivity coefficient between the parietal lobe and the temporal lobe, L ZN Represents the connectivity coefficient between the occipital lobe and temporal lobe brain regions.
[0031] The delta wave frequency band is 1-4Hz, the theta wave frequency band is 4-8Hz, the alpha wave frequency band is 8-13Hz, the beta wave frequency band is 13-30Hz and the gamma wave frequency band is 31-50Hz.
[0032] Specifically, 1) Analyze the connectivity coefficient L between the frontal lobe and parietal lobe brain regions ED :
[0033]
[0034] Where J represents the number of sampling points, i is an imaginary number, and H represents the number of acquisition channels in the frontal lobe brain area. represents the phase of the EEG signal of the hth channel in the frontal lobe at the jth sampling point, K represents the number of acquisition channels in the parietal lobe, Represents the phase of the EEG signal of the kth channel in the parietal brain area at the jth sampling point.
[0035] 2) Analyze the connectivity coefficient L between the frontal lobe and occipital lobe EZ :
[0036]
[0037] Among them, L represents the number of acquisition channels in the occipital lobe brain area, Represents the phase of the EEG signal of the lth channel in the occipital lobe at the jth sampling point.
[0038] 3) Analyze the connectivity coefficient L between the frontal lobe and temporal lobe EN :
[0039]
[0040] Among them, M represents the number of acquisition channels in the temporal lobe brain area, Represents the phase of the EEG signal of the mth channel in the temporal lobe brain area at the jth sampling point.
[0041] 4) Analyze the connectivity coefficient L between the parietal lobe and occipital lobe DZ :
[0042]
[0043] 5) Analyze the connectivity coefficient L between the parietal lobe and temporal lobe DN :
[0044]
[0045] 6) Analyze the connectivity coefficient L between the occipital lobe and temporal lobe ZN :
[0046]
[0047] The control module 31 is also used to input the A-dimensional feature matrix into the trained cognitive dysfunction assessment model for prediction, so as to output the evaluator's cognitive dysfunction level and control the display module 32 to display, and the cognitive dysfunction level includes no cognitive dysfunction level, mild cognitive dysfunction level, moderate cognitive dysfunction level and severe cognitive dysfunction level.
[0048] The cognitive dysfunction assessment model adopts a deep learning model, which includes two feature extraction layers, a fully connected layer and a Softmax classifier in sequence. The two feature extraction layers include a convolution layer, a batch normalization layer and an activation function layer. The number of convolution kernels in the convolution layer of the first feature extraction layer is an integer multiple of the number of convolution kernels in the convolution layer of the second feature extraction layer.
[0049] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that these are only examples, and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.
Claims
1. A cognitive dysfunction assessment system based on virtual reality and EEG signals, characterized in that: It includes VR glasses and a head-mounted EEG acquisition device for the assessor to wear, and a server that interacts with the VR glasses and the head-mounted EEG acquisition device, wherein the server includes a control module and a display module; The control module is used to control the VR glasses to play and display A virtual reality scene videos in sequence with rest intervals for the evaluator to watch, the virtual reality scene videos are videos with sound, the viewing difficulty of the A virtual reality scene videos ranges from none to one, from low to high, and the viewing time of each virtual reality scene video is the same, and in the process of the evaluator watching each virtual reality scene video, the head-mounted electroencephalogram acquisition device is controlled to continuously collect electroencephalogram signals of multiple channels in the frontal lobe brain area, the parietal lobe brain area, the occipital lobe brain area and the temporal lobe brain area at a set sampling frequency, A is a positive integer and A≥2; The control module is also used to perform denoising filtering on the EEG signals of each channel of each brain region corresponding to each virtual reality scene video to obtain the corresponding pre-processed EEG signals; The control module is also used for the preprocessed EEG signals of each channel of each brain region corresponding to the a-th virtual reality scene video: judging whether there are abnormal signals at each sampling point in the preprocessed EEG signals of each channel of each brain region, and taking the preprocessed EEG signals of each channel of each brain region as qualified EEG signals when there are no abnormal signals, and performing abnormal signal processing when there are abnormal signals and taking the EEG signals of each channel of each brain region obtained after the abnormal signal processing as qualified EEG signals, respectively analyzing the variance of the qualified EEG signals of each channel of each brain region, respectively performing Fourier transform on the qualified EEG signals of each channel of each brain region, and analyzing the power spectral density of the δ wave, θ wave, α wave, β wave and γ wave frequency bands of each channel, analyzing the connectivity coefficients between each brain region, and constructing a one-dimensional feature matrix (E f 、E δ 、E θ 、E α 、E β 、E γ , D f , D δ , D θ , D α , D β , D γ , Z f , Z δ , Z θ , Z α , Z β , Z γ 、N f 、N δ 、N θ 、N α 、N β 、N γ , L ED , L EZ , L EN , L DZ , L DN , L ZN ), a is taken as 1-A in sequence, so as to obtain an A-dimensional feature matrix consisting of feature matrices corresponding to A virtual reality scene videos; Among them, E f The average value of the variance of qualified EEG signals of each channel in the frontal lobe brain area, E δ The average value of the delta wave power spectral density of qualified EEG signals of each channel in the frontal lobe brain area, D f The average value of the variance of qualified EEG signals of each channel in the parietal brain area, D δ The average value of the delta wave power spectral density of the qualified EEG signals of each channel in the parietal brain area, Z f Represents the average value of the variance of qualified EEG signals of each channel in the occipital lobe brain area, Z δ The average value of the delta wave power spectral density of the qualified EEG signals of each channel in the occipital lobe brain area, N f Represents the average value of the variance of qualified EEG signals of each channel in the temporal lobe brain area, N δ The average value of the delta wave power spectral density of qualified EEG signals of each channel in the temporal lobe brain area, L ED represents the connectivity coefficient between the frontal lobe and the parietal lobe, L EZ represents the connectivity coefficient between the frontal lobe and the occipital lobe, L EN represents the connectivity coefficient between the frontal lobe and the temporal lobe, L DZ represents the connectivity coefficient between the parietal lobe and the occipital lobe, L DN represents the connectivity coefficient between the parietal lobe and the temporal lobe, L ZN Represents the connectivity coefficient between the occipital lobe and temporal lobe brain regions; The control module is also used to input the A-dimensional feature matrix into the trained cognitive dysfunction assessment model for prediction, so as to output the evaluator's cognitive dysfunction level and control the display module to display, and the cognitive dysfunction level includes no cognitive dysfunction level, mild cognitive dysfunction level, moderate cognitive dysfunction level and severe cognitive dysfunction level.
2. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 1, characterized in that: The control module is also used for the preprocessed EEG signals of each channel of each brain region corresponding to the ath virtual reality scene video: first, the EEG signals of B sampling points from the head end and the EEG signals of C sampling points from the tail end of the preprocessed EEG signals of each channel of each brain region are intercepted to obtain the processed segment EEG signals of each channel of each brain region, and then it is determined whether there are abnormal signals at each sampling point in the processed segment EEG signals of each channel of each brain region. When there are no abnormal signals, the processed segment EEG signals of each channel of each brain region are used as qualified EEG signals. When there are abnormal signals, the abnormal signals are processed and the EEG signals of each channel of each brain region obtained after the abnormal signal processing are used as qualified EEG signals. B and C are positive integers.
3. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 2, characterized in that: The control module is also used to process abnormal signals when there are abnormal signals and use the EEG signals of each channel of each brain region obtained after the abnormal signal processing as qualified EEG signals, specifically: first determine whether there is a situation in which multiple consecutive sampling points in the EEG signals of a certain channel of at least one brain region have abnormal signal amplitudes, and if so, directly delete the EEG signals of the consecutive multiple sampling points in the channel of the brain region with abnormal signal amplitudes and then splice them, and at the same time delete the EEG signals of other channels of the brain region and the corresponding deleted segments of each channel of other brain regions and then splice them, and then determine whether there is a situation in which a single sampling point in the EEG signals of a certain channel of at least one brain region has abnormal signal amplitudes, and if so, further determine whether the single amplitude abnormality of the channel of the brain region is added. After deleting the signals of the common sampling points and deleting the EEG signals of other channels of the brain region and the corresponding deleted points of each channel of other brain regions, whether the time series length of the EEG signal of a certain channel after splicing reaches the set time series length, if it is, the EEG signals after deleting and splicing each single sampling point of each channel of each brain region are regarded as qualified EEG signals, when the time series length of the spliced EEG signals does not reach the set time series length, the single sampling point signal of the channel of the brain region with abnormal amplitude is corrected, and the corrected EEG signals of each channel of each brain region are regarded as qualified EEG signals, and when there is no abnormal amplitude of the single sampling point signal, the EEG signals after deleting and splicing each continuous multiple sampling points of each channel of each brain region are regarded as qualified EEG signals; When there are no consecutive abnormal signal amplitudes of multiple sampling points in the EEG signal of a certain channel in a certain brain region, it indicates that there is a single abnormal signal amplitude at a single sampling point in the EEG signal of a certain channel in at least one brain region. The EEG signal of the single abnormal amplitude sampling point in the channel of the brain region is directly deleted and then spliced. At the same time, the EEG signals of the corresponding deleted points of other channels of the brain region and each channel of other brain regions are deleted and then spliced. The EEG signals after deleting and splicing each single sampling point in each channel of each brain region are used as qualified EEG signals.
4. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 3, characterized in that: The control module is also used to correct the signal of a single abnormal amplitude sampling point of the channel in the brain region: the average value of the amplitudes of the previous sampling point and the next sampling point of the abnormal amplitude sampling point of the channel in the brain region is replaced by the new amplitude of the abnormal amplitude sampling point.
5. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 1, characterized in that: The control module is also used to analyze the connectivity coefficient L between the frontal lobe brain area and the parietal lobe brain area ED : Where J represents the number of sampling points, i is an imaginary number, and H represents the number of acquisition channels in the frontal lobe brain area. represents the phase of the EEG signal of the hth channel in the frontal lobe at the jth sampling point, K represents the number of acquisition channels in the parietal lobe, represents the phase of the EEG signal of the kth channel in the parietal brain area at the jth sampling point; The control module is also used to analyze the connectivity coefficient L between the frontal lobe brain area and the occipital lobe brain area EZ : Among them, L represents the number of acquisition channels in the occipital lobe brain area, represents the phase of the EEG signal of the lth channel in the occipital lobe at the jth sampling point; The control module is also used to analyze the connectivity coefficient L between the frontal lobe brain area and the temporal lobe brain area EN : Among them, M represents the number of acquisition channels in the temporal lobe brain area, represents the phase of the EEG signal of the mth channel in the temporal lobe at the jth sampling point; The control module is also used to analyze the connectivity coefficient L between the parietal lobe brain area and the occipital lobe brain area DZ : The control module is also used to analyze the connectivity coefficient L between the parietal lobe brain area and the temporal lobe brain area DN : The control module is also used to analyze the connectivity coefficient L between the occipital lobe brain region and the temporal lobe brain region ZN : .
6. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 1, characterized in that: The VR glasses are provided with a left eye tracking camera and a right eye tracking camera; The control module is also used to control the left eye tracking camera to collect left eye movement video images and control the right eye tracking camera to collect right eye movement video images during the process of the evaluator watching each virtual reality scene video; The control module is also used for: for the left eye movement video image and the right eye movement video image corresponding to the ath virtual reality scene video: positioning the pupil center of each frame of the left eye movement video image, obtaining the change of the left pupil center coordinate in time sequence, analyzing the change values of the left pupil center coordinate twice before and after, and calculating the variance X of the change value of the left pupil center coordinate 左 , locate the pupil center of each frame of the right eye movement video image, obtain the change of the right pupil center coordinate in time sequence, analyze the change values of the right pupil center coordinate twice before and after, and calculate the variance X of the change value of the right pupil center coordinate 右 , construct a one-dimensional feature matrix (E f 、E δ 、E θ 、E α 、E β 、E γ , D f , D δ , D θ , D α , D β , D γ , Z f , Z δ , Z θ , Z α , Z β , Z γ 、N f 、N δ 、N θ 、N α 、N β 、N γ , L ED , L EZ , L EN , L DZ , L DN , L ZN , X 左 , X 右 ), a is taken as 1-A in sequence, so as to obtain an A-dimensional feature matrix composed of feature matrices corresponding to A virtual reality scene videos.
7. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 1, characterized in that: The VR glasses are provided with a posture sensor; The control module is also used to control the posture sensor to collect head movement during the process of the evaluator watching each virtual reality scene video; The control module is also used for the head movement corresponding to the ath virtual reality scene video: performing head positioning for each collected head movement, obtaining the change of the head fixed point coordinates in time sequence, analyzing the change values of the head fixed point coordinates twice before and after, and calculating the variance X of the change value of the head fixed point coordinates 头 , construct a one-dimensional feature matrix (E f 、E δ 、E θ 、E α 、E β 、E γ , D f , D δ , D θ , D α , D β , D γ , Z f , Z δ , Z θ , Z α , Z β , Z γ 、N f 、N δ 、N θ 、N α 、N β 、N γ , L ED , L EZ , L EN , L DZ , L DN , L ZN , X 左 , X 右 , X 头 ), a is taken as 1-A in sequence, so as to obtain an A-dimensional feature matrix composed of feature matrices corresponding to A virtual reality scene videos.
8. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 1, characterized in that: The cognitive dysfunction assessment model adopts a deep learning model, which includes two feature extraction layers, a fully connected layer and a Softmax classifier in sequence. The two feature extraction layers both include a convolution layer, a batch normalization layer and an activation function layer. The number of convolution kernels of the convolution layer of the first feature extraction layer is an integer multiple of the number of convolution kernels of the convolution layer of the second feature extraction layer.
9. The cognitive dysfunction assessment system based on virtual reality and EEG signals as claimed in claim 1, characterized in that: The delta wave frequency band is 1-4Hz, the theta wave frequency band is 4-8Hz, the alpha wave frequency band is 8-13Hz, the beta wave frequency band is 13-30Hz and the gamma wave frequency band is 31-50Hz.
Citation Information
Patent Citations
Anomalous amplitude suppression method and apparatus
CN104932017A
Object cognitive evaluation method, device, equipment and readable storage medium
CN110859616A
Emotion cognition method based on electroencephalogram signal feature analysis
CN112603332A
Mild cognitive impairment recognition method and device, computer equipment and storage medium
CN113243891A
Abnormal sampling point processing method and device based on waveform amplitude calculation feedback analysis
CN114089034A
