Sample data enhancement-based steady-state visual evoked potential electroencephalogram feature decoding method

Abstract

The invention discloses a steady-state visual evoked potential electroencephalogram feature decoding method based on sample data enhancement, which can more accurately simulate dynamic features of brain waves by capturing and reproducing frequency and phase features of original SSVEP (Steady-State Visual Evoked Potential) signals through an aliasing matrix. According to the method, the similarity between the composite signal and the real SSVEP signal is enhanced through the shifting operation. The shift operation keeps the frequency component of the signal unchanged by adjusting the position of the synthetic signal on the time axis, and ensures that the time characteristic of the synthetic signal is more consistent with that of the real SSVEP signal. According to the invention, the diversity and the classification accuracy of the synthesized data are ensured by respectively synthesizing the data for multiple categories. The method is suitable for a multi-classification SSVEP-BCI system, the classification performance and adaptability in the computer feature decoding process can be effectively improved, and the accuracy and efficiency of computer feature decoding can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of brain-computer interface technology, and further relates to a method for decoding steady-state visual evoked potential EEG features based on sample data enhancement. The present invention can be used for EEG feature decoding in a brain-computer interface system. Background Art

[0002] Brain Computer Interface (BCI) refers to a direct connection between the human or animal brain and an external device to achieve information exchange between the brain and the device. Steady-State Visual Evoked Potential (SSVEP) has a relatively high signal-to-noise ratio, requires less training, and provides the highest information transmission rate (ITR). Therefore, SSVEP is widely used in the EEG feature decoding of vision-related BCI systems.

[0003] Tianjin University proposed a low-training-cost EEG feature decoding method for steady-state visual evoked potential in its patent application "Low-training-cost EEG feature decoding method for steady-state visual evoked potential" (application number CN 202310981789.2 application publication date CN 117009766.A application date 2023-11-07). The implementation steps of the method include: (1) collecting the original steady-state visual evoked potential signal of the subject as training data; (2) using a filter group to preprocess the original steady-state visual evoked potential Xn to obtain a preprocessed steady-state visual evoked potential; (3) dividing the preprocessed steady-state visual evoked potential into Pn sub-periods, integrating the spatial filters according to the spatial filters of the mth sub-frequency band, and obtaining an integrated spatial filter pair. Average, obtain a copy and connect multiple synthesis; (4) obtain a steady-state visual evoked potential template signal of the required length; (5) identifying the unknown steady-state visual evoked potential K by integrating the spatial filter and the steady-state visual evoked potential template signal. Although this method solves the technical problem that the recognition performance drops significantly when there is less training data when using the spatial filtering algorithm based on individual training data, especially when there is only single-trial training data, the recognition performance is very poor, and achieves the technical effect of improving the recognition accuracy of unknown visual stimuli. However, this method still has three shortcomings: first, the lack of sufficient learning of the personalized signal characteristics of the subjects leads to limitations such as a limited number of recognizable commands, a long induction time, and poor recognition performance. Since this method fails to fully capture the unique signal characteristics of each subject, it affects the recognition efficiency and accuracy, and cannot achieve ideal information transmission, accuracy, and efficiency in BCI; second, the spatial filtering algorithm based on individual training data requires enough data to obtain an effective spatial filter and individual SSVEP template. Insufficient training data for each stimulus frequency will lead to unreliable covariance estimation of the spatial filter, and the signal-to-noise ratio of the individual SSVEP template obtained by averaging multiple trials is low, which will significantly reduce the SSVEP recognition performance; third, when generating synthetic samples that are very similar to the real SSVEP data, the time-locked and phase-locked characteristics of the SSVEP signal are not considered. Ignoring these characteristics will result in a large difference between the synthetic samples and the real samples, thereby affecting the training effect and performance of the recognition model. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and to provide a method for decoding steady-state visual evoked potential EEG features based on sample data enhancement, aiming to solve the problem that the simple sample transformation method in the prior art for EEG feature decoding cannot accurately simulate the dynamic characteristics of brain waves, the signal is distorted, and only a single category of data can be generated.

[0005] The idea of ​​achieving the purpose of the present invention is that the present invention uses individual calibration data to estimate the aliasing matrix, and the aliasing matrix can be used to accurately capture and reproduce the frequency and phase characteristics of the steady-state visual evoked potential EEG signal of each individual. The aliasing matrix ensures that the synthesized signal is highly consistent with the original EEG signal not only in terms of numerical value but also in terms of physiological characteristics, thereby avoiding signal distortion caused by attribute simplification in traditional methods. The present invention uses the periodicity of the SSVEP signal to shift the synthesized reference signal multiple times, solving the problem of signal phase and frequency-specific distortion that may be caused by the enhancement method in the prior art. The shift operation generates signal samples with different time offsets by periodically moving the synthesized signal. The diversity of the signals in the data set is increased, and the shift operation can ensure that the frequency characteristics of the signal remain unchanged, thereby maintaining the frequency specificity of the signal and avoiding classification errors and interpretation errors caused by frequency deviations. The amount of training data is expanded, and the overall stability and reliability of the signal processing system are improved. The present invention splices the shifted synthetic signal with the original signal, increases the amount of synthetic data, expands the training set, overcomes the problems of small training set data volume and weak model generalization ability in the prior art, and improves the accuracy of EEG feature decoding.

[0006] The technical solution for achieving the purpose of the present invention is as follows:

[0007] Step 1, generate a sine-cosine reference matrix, the rows of which alternately represent the amplitudes of the sine and cosine of the SSVEP EEG signal harmonics, and the columns of which represent the sampling moments of the SSVEP EEG signal harmonics; estimate the error between each sampling moment of the SSVEP EEG signal harmonics and each element of the sine-cosine reference matrix to obtain an aliasing matrix; estimate the optimal aliasing matrix by the minimum mean square error criterion, and multiply the optimal aliasing matrix by the reference matrix to obtain the reconstructed source signal;

[0008] Step 2, shifting the reconstructed source signal, the number of shifts is equal to the number of cycles of the source signal, and the length of each shift is equal to the length of a source signal cycle; the signal after each shift is taken as a sample;

[0009] Step 3, concatenate all the shifted samples with the SSVEP EEG signals to form a training set;

[0010] Step 4: Use the spatial filtering algorithm to calculate the weight value of the spatial filter for each sample in the training set, calculate the correlation coefficient by weighted sum of the spatial filter weight value and the original sample signal, and select the stimulation frequency corresponding to the maximum correlation coefficient as the target stimulation source, i.e., decoding.

[0011] Compared with the prior art, the present invention has the following advantages:

[0012] First, the present invention uses an aliasing matrix estimation method to synthesize samples similar to real SSVEP signals. This method captures and reproduces the frequency and phase characteristics of the original SSVEP signal through the aliasing matrix, and can more accurately simulate the dynamic characteristics of brain waves. Compared with the simple sample transformation method in the prior art, it overcomes the defect of being unable to accurately simulate the dynamic characteristics of brain waves, so that the present invention significantly improves the authenticity and effectiveness of the synthesized signal, making it suitable for SSVEP-BCI systems that have high requirements for the dynamic characteristics of the signal, and optimizes the accuracy and efficiency of computer feature decoding.

[0013] Second, the present invention enhances the similarity between the synthetic signal and the real SSVEP signal through a shift operation. The shift operation adjusts the position of the synthetic signal on the time axis to keep the frequency component of the signal unchanged, while ensuring that the time characteristics of the synthetic signal are more consistent with the real SSVEP signal. This precise time matching improves the simulation and decoding capabilities of EEG signals, thereby improving the system's recognition and response efficiency to EEG activities, and effectively reducing signal distortion problems that may be caused by data enhancement methods. These improvements enable the present invention to significantly improve the signal processing performance in the decoding process, and is particularly suitable for use in SSVEV-BCI systems that have strict requirements on high signal-to-noise ratio and data authenticity.

[0014] Third, the present invention overcomes the defect of only generating single-category data in the prior art by synthesizing data for multiple categories separately. In the SSVEP-BCI system, different categories of SSVEP signals correspond to different visual stimulation frequencies. The present invention synthesizes data for each category separately to ensure that the generated samples can cover the characteristics of all target categories. This method ensures the diversity and classification accuracy of the synthesized data, making the present invention suitable for multi-classification SSVEP-BCI systems, and can effectively improve the classification performance and adaptability in the computer feature decoding process. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0016] The following is combined with Figure 1 The implementation steps of the present invention are further described in detail with reference to the accompanying drawings.

[0017] Step 1, generate a sine-cosine reference matrix, the rows of which alternately represent the amplitudes of the sine and cosine of the SSVEP EEG signal harmonics, and the columns of which represent the sampling moments of the SSVEP EEG signal harmonics. Estimate the error between each sampling moment of the SSVEP EEG signal harmonics and each element of the sine-cosine reference matrix to obtain an aliasing matrix. The optimal aliasing matrix is ​​estimated by the minimum mean square error criterion, and the optimal aliasing matrix is ​​multiplied by the reference matrix to obtain the reconstructed source signal.

[0018] The sample data of the embodiment of the present invention is derived from the benchmark data set of Tsinghua University.

[0019] Display N on the screen f The frequencies are The original steady-state visual evoked potential signals of the subjects looking at these visual stimuli are collected as training data. The subjects’ gaze frequency is g $ The original steady-state visual evoked potential of the subject's multi-electroencephalogram channels collected when the visual stimulus is X $ , in Represents the number field, N ' Indicates the number of EEG signal channels, N ( Indicates the number of sampling points, N ) Indicates the number of sampling trials.

[0020] The average template of the individual calibration data is calculated as follows:

[0021]

[0022] in, The subject's gaze frequency is f $ The stimulus signal at the time is in the time window [0,t w ] and keep it as t w represents the length of the time window, and h represents the sequence number of individual calibration times.

[0023] In order to capture the periodicity of the SSVEP response, a sine-cosine reference matrix corresponding to each subject at different gaze frequencies was constructed as follows:

[0024]

[0025] Among them, R n The subject's gaze frequency is f n The sine-cosine reference matrix, N H Indicates the total number of harmonic frequencies, which is set to 5 in the embodiment of the present invention. * Indicates the number of sampling points in the time window, N s =tw ×fs,t w Indicates the time window length, its value range is 0.3~1s, fs indicates the sampling frequency, t indicates the sampling time, t=1,2,…,N s .

[0026] The error between each sampling moment of the harmonics of the SSVEP EEG signal and each element of the sine and cosine reference matrix is ​​estimated to obtain the aliasing matrix.

[0027] To improve the accuracy of the estimate, a mixing matrix correlating the sub-band SSVEP responses with the recorded EEG channels was constructed. The minimum mean square error (MMSE) optimization method is used to estimate the optimal aliasing matrix.

[0028] The minimum mean square error criterion for individual calibration EEG data refers to the optimization value after the error between the output signal and the expected signal is minimized by adjusting the weight of the filter at each frequency by minimizing the mean square error.

[0029] The optimal aliasing matrix is ​​as follows:

[0030]

[0031] in, The subject's gaze frequency is f n The aliasing matrix A is estimated using the minimum mean square error criterion of the individual calibrated EEG data. $ The optimal aliasing matrix is Indicates A $ Take the minimum operation in the domain, Indicates that the Frobenius norm is used to measure the matrix difference.

[0032] The optimal aliasing matrix is ​​used to reconstruct the source signal, which is obtained by the following formula:

[0033]

[0034] in, represents the optimal aliasing matrix, R n represents the sine-cosine reference matrix.

[0035] Step 2: Shift the reconstructed source signal, the number of shifts is equal to the number of cycles of the source signal, and the length of each shift is equal to the length of a source signal cycle. The signal after each shift is taken as a sample.

[0036] SSVEP responses exhibit inherent periodic properties. Exploiting this property, synthetic reference signals with periodic properties can also be generated. By applying a specific shifting step size to these synthetic reference signals, the resulting spectrum retains the shifted fundamental and harmonic components while significantly attenuating other frequency components. Crucially, when the shifting frequency matches the target stimulus frequency, the shifted signal retains the SSVEP properties. Through the process of shifting the artificially synthesized reference signal, multiple signal trials that retain the SSVEP properties can be generated.

[0037] The length of the artificial synthetic signal cycle of the subject is calculated according to the following formula:

[0038]

[0039] Among them, k n Represents the period length of the artificial synthetic signal of the subject.

[0040] For the original N t The number of trials of the source signal can be expanded to N through the time shift process. a For the i-th shift iteration, where i = 1, 2, ..., N a , displacement length L i , L i =i×k n .

[0041] Source signal S n It consists of the following two parts:

[0042]

[0043] in, Indicates signal S n The last L i A sample of Indicates S n The remaining samples.

[0044] The amount Move to signal In front of it, a new sample signal Z is generated n ,

[0045] Step 3: Concatenate all shifted samples with the SSVEP EEG signals to form a training set.

[0046] The new signal Z n With X n Concatenate to form training data X n , X n =[Z n ,X n ].

[0047] N t The results of the trials are obtained by shifting the data N a times, and the length of each shift is L i This process produces a new, expanded dataset This dataset includes the original trials and N a additional trials, a total of N aug Trials, N aug =N t +N a .

[0048] The shifted signal samples generated by the above steps will form a training data set together with the original signal. This not only solves the problem of small training data volume, but also improves the generalization ability and accuracy of the model in practical applications.

[0049] Step 4: Use the spatial filtering algorithm to calculate the weight value of the spatial filter for each sample in the training set, calculate the correlation coefficient by weighted sum of the spatial filter weight value and the original sample signal, and select the stimulation frequency corresponding to the maximum correlation coefficient as the target stimulation source, i.e., decoding.

[0050] State-of-the-art trained spatial filtering algorithms have been widely adopted in SSVEP-BCI because they can effectively extract discriminative spatial filters from EEG signals. By exploiting the periodicity and synchronization of SSVEP responses, training-based algorithms can identify optimal spatial filters that maximally capture task-related SSVEP components. These spatial filters minimize the effects of noise and non-task-related brain activity while maximizing the cross-trial reproducibility of SSVEP signals.

[0051] The training data set X n Decomposed into N2 subband components, using filter bank analysis expressed as Then Perform task-related component analysis to obtain the spatial filter corresponding to the nth stimulus source

[0052] Use the spatial filter obtained through training The test data is decomposed into Y subband components and expressed as Y using filter bank analysis. j (j=1,2,…,N2), and average the training data after multiple trials Perform spatial filtering, and then calculate the correlation coefficient between the two after spatial filtering. The calculation formula is as follows:

[0053]

[0054] in, represents the correlation coefficient of the jth subband, ρ represents the calculation of the two-dimensional correlation coefficient, Y j represents the subband component, represents the spatial filter of the j-th subband, represents the training data of the j-th subband.

[0055] Calculate the collective correlation coefficient γ $ as follows:

[0056]

[0057] Among them, γ $ represents the correlation coefficient, and ω(j) represents the spatial filter weight.

[0058] Target stimulation frequency The acquisition is as follows: The stimulation frequency corresponding to the maximum correlation coefficient is selected as the target stimulation source to complete the EEG signal decoding.

Claims

1. A method for decoding steady-state visual evoked potential EEG features based on sample data enhancement, It is characterized in that The aliasing matrix is ​​estimated using individual calibration EEG data, and the reconstructed source signal is shifted multiple times using the periodicity of the SSVEP signal to generate signal samples with different time offsets, and the shifted synthetic signal is spliced ​​with the original signal to increase the amount of synthesized data; the steps of the decoding method include the following: Step 1, generate a sine-cosine reference matrix, the rows of which alternately represent the amplitudes of the sine and cosine of the harmonics of the SSVEP EEG signal, and the columns of which represent the sampling moments of the harmonics of the SSVEP EEG signal; Estimate the error between each sampling moment of the harmonics of the SSVEP EEG signal and each element of the sine-cosine reference matrix to obtain the aliasing matrix; estimate the optimal aliasing matrix through the minimum mean square error criterion, multiply the optimal aliasing matrix with the sine-cosine reference matrix to obtain the reconstructed source signal; Step 2, shifting the reconstructed source signal, the number of shifts is equal to the number of cycles of the source signal, and the length of each shift is equal to the length of a source signal cycle; taking the signal after each shift as a sample; Step 3, concatenate all the shifted samples with the SSVEP EEG signals to form a training set; Step 4: Use the spatial filtering algorithm to calculate the weight value of the spatial filter for each sample in the training set, calculate the correlation coefficient by weighted sum of the spatial filter weight value and the original sample signal, and select the stimulation frequency corresponding to the maximum correlation coefficient as the target stimulation source, i.e., decoding.

2. The method for decoding steady-state visual evoked potential electroencephalogram features based on sample data enhancement according to claim 1, It is characterized in that The sine-cosine reference matrix described in step 1 is as follows: Among them, R n Indicates that the fixation frequency of the nth subject is f n The sine-cosine reference matrix, N H Indicates the total number of harmonic frequencies, which is set to 5 in the embodiment of the present invention. s Indicates the number of sampling points in the time window, N s =t w ×fs,t w Indicates the time window length, its value range is 0.3~1s, fs indicates the sampling frequency, t indicates the sampling time, t=1,2,…,N s .

3. The method for decoding steady-state visual evoked potential electroencephalogram features based on sample data enhancement according to claim 2, It is characterized in that The minimum mean square error criterion for the individual calibrated EEG data described in step 1 refers to the optimization value after minimizing the error between the output signal and the expected signal by adjusting the weight of the filter at each frequency by minimizing the mean square error.

4. The method for decoding steady-state visual evoked potential electroencephalogram features based on sample data enhancement according to claim 3, It is characterized in that The optimal aliasing matrix described in step 1 is as follows: in, The subject's gaze frequency is f n The aliasing matrix A is estimated using the minimum mean square error criterion of the individual calibrated EEG data. " The optimal aliasing matrix is Indicates A " Take the minimum operation in the domain, Indicates that the Frobenius norm is used to measure the matrix difference. express In the time window [0,t w ] internal segment, t w Indicates the time window length, and its value range is 0.3~1s. Represents the average template of the individual calibration data.

5. The method for decoding steady-state visual evoked potential electroencephalogram features based on sample data enhancement according to claim 4, It is characterized in that The average template of the individual calibration data is obtained by the following formula: Among them, N t represents the total number of individual calibrations, h represents the sequence number of individual calibrations, and X ) The subject's gaze frequency is f ) The original steady-state visual evoked potentials of the subjects' multi-electroencephalogram channels collected when the visual stimulation R (·) Represents number, N c Indicates the number of channels of EEG signal, N d Indicates the number of sampling points, N d =t w ×fs, fs represents the sampling frequency, N * Indicates the number of sampling trials.

6. The method for decoding steady-state visual evoked potential electroencephalogram features based on sample data enhancement according to claim 5, It is characterized in that The steps of shifting the synthesized reference signal by utilizing the periodicity of the SSVEP signal in step 2 are as follows: The first step is to calculate the period length of the subject's artificial synthetic signal according to the following formula: Among them, k n represents the period length of the artificial synthetic signal of the subject; The second step is to generate the source signal S n : in, Indicates S n The remaining samples in Indicates signal S n The last L i samples of The third step is to divide the Move to signal In front of it, a new sample signal Z is generated n , Step 4: Generate a new signal Z n With X n Concatenate to form training data X n , X n =[Z n ,X n ].

7. The method for decoding steady-state visual evoked potential electroencephalogram features based on sample data enhancement according to claim 6, It is characterized in that The step 3 of concatenating the shifted synthesized signal with the original signal means that N t The results of the trials are obtained by shifting the data N a times, and the length of each shift is L i , resulting in a new extended dataset This dataset includes the original trials and N a additional trials, a total of N aug Trials, N aug =N t +N a .

8. The method for decoding steady-state visual evoked potential electroencephalogram features based on sample data enhancement according to claim 1, It is characterized in that The spatial filtering algorithm described in step 4 refers to using the spatial filter obtained through training The test data is decomposed into Y subband components and expressed as Y using filter bank analysis. j (j=1,2,…,N j ), and average the training data after multiple trials Perform spatial filtering, then calculate the correlation coefficient between the two after spatial filtering, and select the stimulation frequency corresponding to the maximum correlation coefficient as the target stimulation source.

Images