A method and device for processing and screening SSVEP signals based on a brain-computer interface

Abstract

The application provides a method and device for SSVEP signal processing and screening based on a brain-computer interface, which comprises data import and parameter setting, data denoising, time domain mean calculation of the SSVEP signal collected by multiple channels to remove physiological artifacts, cyclic processing of each data block, data preprocessing and frequency spectrum data screening steps, baseline drift removal, notch filtering, band pass filtering and fast Fourier transform of each data block, result display and output of corresponding information according to the screening condition. The data preprocessing is processed by time domain and frequency domain filters, which can simply and effectively remove physiological artifacts of the SSVEP signal and improve the quality of the original SSVEP signal. The frequency screening step overcomes the defects of time-consuming and laborious manual screening, adjusts the function parameters according to the characteristics of the SSVEP signal, the fundamental harmonic characteristics and individual differences, and quickly screens and retains the effective SSVEP signal, thereby providing strong support for subsequent tasks such as a high-quality data set.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of brain-computer interface and signal processing, and particularly relates to a method and device for SSVEP signal processing and screening based on a brain-computer interface. BACKGROUND

[0002] Brain-computer interface (BCI) has become a popular research direction in the field of human-computer interaction and medical rehabilitation. BCI based on electroencephalogram (EEG) is concerned due to its convenience, low cost and non-invasiveness. In the EEG paradigm, steady-state visual evoked potential (SSVEP) is one of the popular paradigms due to its high signal-to-noise ratio and low training time.

[0003] In the BCI system based on SSVEP, the quality of SSVEP signal data and feature extraction are crucial. However, SSVEP signal acquisition is interfered by power frequency and physiological artifacts, and signal processing methods mainly include time domain, frequency domain and time-frequency domain. Based on the characteristic that the frequency of SSVEP signal is consistent with the response frequency, existing technologies mainly focus on developing algorithms in the frequency domain, mainly through analyzing the frequency characteristics of SSVEP signal to extract the features of the signal, and then performing tasks such as classification, target recognition, encoding and decoding.

[0004] In related prior art, power spectral density analysis (PSDA) identifies the main frequency components by calculating the power spectral density of SSVEP signal, uses FFT to convert the signal, senses the peak value at the stimulation frequency and compares it with the threshold value, and the shortcomings are low classification accuracy when the noise is strong or the signal is unstable, and the spatial dependence of multi-channel signal is ignored; canonical correlation analysis (CCA) identifies SSVEP signal by finding the maximum correlation between the EEG signal and the reference signal, and uses multi-channel spatial information to enhance robustness, but does not consider the non-base frequency components of the signal, which may miss important spectral information, resulting in decreased classification robustness; filter bank CCA (FBCCA) decomposes SSVEP signal into different frequency band sub-components and performs CCA algorithm, and the weighted average correlation coefficient is used to obtain the classification result, and the generalization ability for different users needs to be further verified.

[0005] The above prior art has common shortcomings, such as not considering power frequency interference and baseline shift, lacking flexibility, and not considering the quality of original signal data and mainly using manual screening.

[0006] Therefore, it is of great significance to propose a method and device for SSVEP signal processing and screening based on a brain-computer interface. SUMMARY

[0007] In order to solve the problems of complex environmental noise interference, individual difference neglect, low signal screening efficiency and data quality restriction of subsequent tasks in the existing SSVEP signal processing technology, the present application provides a method and device for SSVEP signal processing and screening based on brain-computer interface to solve the above technical defects.

[0008] In the first aspect, the present application provides a method for SSVEP signal processing and screening based on brain-computer interface, which comprises the following steps:

[0009] Data import and parameter setting, importing the acquired SSVEP signal data, and setting the response frequency, the window size of local maximum value and the offset value of response frequency;

[0010] Data denoising, time domain mean calculation is performed on the SSVEP signal collected by multiple channels to remove physiological artifacts;

[0011] Cyclic processing of each data block, including data preprocessing steps, baseline drift removal, notch filtering, band pass filtering and fast Fourier transform are performed on each data block; the frequency spectrum data screening step is also included, the local maximum value of the frequency spectrum amplitude and its corresponding frequency are extracted within the set window, and the frequency which is consistent with the set response frequency and its second harmonic frequency and third harmonic frequency and within the allowed offset tolerance value is screened out;

[0012] The results show that the corresponding information is output according to the screening condition, if the screening condition is met, the data block and its characteristic frequency which meet the condition are output, and the frequency spectrum graph of all the database which meets the condition is displayed; if not, "no block found" is output.

[0013] Preferably, in the frequency spectrum data screening step, local maximum value screening is included.

[0014] For frequency spectrum data, a fixed window size is set, the local maximum value of the amplitude and its corresponding frequency are extracted within each window, assuming that the window size is N, within the window Wi, the following is found: Where A(f) is the frequency spectrum amplitude, f max and A max are the frequency and the amplitude of the local maximum value respectively.

[0015] Further preferably, frequency screening is also included.

[0016] For the extracted local maximum value, the frequency consistent with the characteristics of SSVEP signal is screened out, which means that the selected signal frequency should be consistent with the set response frequency fr and its second harmonic frequency 2fr and third harmonic frequency 3fr, and the frequency selection condition is as follows: |f max -f r| <Af, where Af is the allowed offset tolerance, by comparing the maximum amplitude at the response frequency and its harmonic frequencies, only the frequencies that meet the conditions are retained.

[0017] Preferably, in the data preprocessing step, the baseline drift is removed by a low-pass filter with a cutoff frequency of 3HZ to stabilize the baseline, and the Butterworth filter is designed as follows: Where H LP (s) represents the transfer function of the low-pass filter; s represents a complex variable, which is used to describe the frequency characteristics of the system in the Laplace transform; ω c represents the cutoff frequency of the low-pass filter, the cutoff frequency ω c = 2π x 3 rad / s; N represents the order of the filter, N = 1; a1, a1, …, a N represent the coefficients of the filter.

[0018] Preferably, in the data preprocessing step, the notch filter is used to remove the power frequency interference by using a 4th order notch filter with a center frequency of 50HZ, avoiding the interference of device noise of power supply and appliances, and the formula is as follows: Where H notch (s) represents the transfer function of the notch filter; s is a complex variable, which is used to describe the frequency characteristics of the system; ω n represents the center frequency of the notch filter, the center frequency ω n = 2π x 50; ζ represents the damping ratio of the filter.

[0019] Preferably, in the data preprocessing step, the band-pass filter is a 4th order band-pass filter with a frequency range from 4Hz to 40Hz, which filters out other noise except the fundamental wave and harmonic wave, thereby retaining the target frequency, and the formula is as follows: Where H bandpass (s) represents the transfer function of the band-pass filter; s is a complex variable, which is used to describe the frequency characteristics of the system; ω l represents the low cutoff frequency of the band-pass filter, ω l = 2π x 4 rad / s; ω h represents the high cutoff frequency of the band-pass filter, ω h = 2π x 40 rad / s.

[0020] Preferably, in the data preprocessing step, the total process of data preprocessing is as follows:

[0021] y(n) = L -1 [L(x(n)) H Lp (S) H noth (S) H bandpass (S)]

[0022] Y(k) = FFT(y(n)), k = 0, 1, 2, …, N-1

[0023] wherein x(n) is the original input signal sequence, n represents the discrete time point of the signal; y(n) is the signal sequence after a series of filtering processing; L(·) and L -1 (·) represent linear transformation and its inverse transformation between different domains of the signal respectively; FFT(·) represents fast Fourier transform, which is used to convert the time domain signal y(n) to the frequency domain to obtain the frequency domain signal Y(k), wherein k represents the frequency index, and N is the length of the input signal.

[0024] Preferably, the response frequency fr is set as [6.25, 7.75, 9.25, 10.75, 12.25].

[0025] In a third aspect, an embodiment of the present application provides a device for SSVEP signal processing and screening based on brain-computer interface, comprising:

[0026] A data import module configured to import the acquired SSVEP signal data;

[0027] A parameter setting module configured to set the response frequency, the window size of the local maximum value, and the offset value of the response frequency;

[0028] A data denoising module configured to perform time domain mean calculation on the multi-channel acquired SSVEP signal to remove physiological artifacts;

[0029] A data block processing module configured to cyclically process each data block, comprising: a data preprocessing submodule configured to perform baseline drift removal, notch filtering, band-pass filtering, and fast Fourier transform on each data block; a spectrum data screening submodule configured to extract the local maximum value of the spectrum amplitude and the corresponding frequency within the set window, and screen out the frequency that is consistent with the set response frequency and its second harmonic frequency and third harmonic frequency and within the allowed offset tolerance value;

[0030] A result display module configured to output corresponding information according to the screening condition, and output the data block and its characteristic frequency that meet the condition and display the spectrum graph of all the databases that meet the condition if the screening condition is met, and output "no block found" if the screening condition is not met.

[0031] In a third aspect, an embodiment of the present application provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the implementations of the first aspect.

[0032] Compared with the prior art, the present application has the following advantages:

[0033] The technical scheme of the present application mainly includes data preprocessing and frequency screening. The data preprocessing removes physiological artifacts of the SSVEP signal and noise interference in the actual working environment through time domain and frequency domain filter processing, thereby improving the quality of the original SSVEP signal. The frequency screening step overcomes the time-consuming and laborious shortcomings of the previous manual screening, and according to the SSVEP signal characteristics and the fundamental harmonic characteristics, the function parameters are adjusted according to individual differences, the effective SSVEP signal is quickly screened and reserved, thereby providing strong support for subsequent tasks such as high-quality data set. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings are included to provide a further understanding of embodiments and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments and, together with the description, serve to explain the principles of the application. Other embodiments and many of the intended advantages of the present application will be readily appreciated as the same becomes better understood by reference to the following detailed description. The elements of the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding similar parts.

[0035] Figure 1 A flowchart of the method of SSVEP signal processing and screening based on brain-computer interface for embodiments of the present application;

[0036] Figure 2 A flowchart of local maximum screening for embodiments of the present application;

[0037] Figure 3 A flowchart of the program flow of frequency screening for embodiments of the present application;

[0038] Figure 4 A general scheme flowchart for a specific embodiment of the present application;

[0039] Figure 5 A time domain and frequency domain diagram of data preprocessing for embodiments of the present application;

[0040] Figure 6 A screening result diagram for a certain experiment for embodiments of the present application;

[0041] Figure 7 A comparison diagram of pre-screening and post-screening results for embodiments of the present application;

[0042] Figure 8 An architecture diagram of the device for SSVEP signal processing and screening based on brain-computer interface for embodiments of the present application;

[0043] Figure 9 A structure diagram of a computer device suitable for implementing the electronic device of the embodiments of the present application. DETAILED DESCRIPTION

[0044] The application will be described in further detail below with reference to the drawings and embodiments. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not intended to limit the application. In addition, it should be noted that only parts related to the application are shown in the drawings for ease of description.

[0045] It should be noted that the embodiments in the application and the features in the embodiments can be combined with each other without conflict. The application will be described in detail below with reference to the drawings and embodiments.

[0046] The embodiments of the application aim to solve several deficiencies in existing SSVEP signal processing techniques. The main problems include:

[0047] 1. Noise interference in complex environments: the existing technology has limited performance in high noise or complex interference environments, and the signal recognition and classification ability is affected;

[0048] 2. Ignoring individual differences: existing algorithms cannot well adjust parameters for individual differences;

[0049] 3. Low efficiency of signal screening: the signal screening link is lacking in the current data processing flow, and the experimenter often uses a time-consuming manual method for screening;

[0050] 4. Data quality constraints on subsequent tasks: high-quality data sets are crucial for classification and target recognition tasks combined with machine learning and deep learning, but existing technologies are often insufficient in this regard.

[0051] To address these issues, the embodiments of the application combine time domain and spectral analysis techniques to suppress physiological artifacts and overcome noise interference in experimental environments (such as low frequency and power frequency interference), improving signal quality. By presetting adjustable parameters, the problem of individual differences is solved. By introducing the fundamental and harmonic characteristics of the signal and combining the characteristics of the SSVEP signal frequency and response frequency, the application realizes efficient screening of SSVEP signals, significantly reducing screening time and improving signal analysis efficiency and data set quality.

[0052] Overall, the technical solution of the application aims to provide an improved SSVEP signal processing method and device that can efficiently utilize multi-channel information in complex environments, adapt to individual differences, achieve real-time and accurate signal processing and data screening, and provide high-quality data sets for subsequent tasks combined with deep learning algorithms.

[0053] In a first aspect, the embodiments of the application disclose a method for SSVEP signal processing and screening based on a brain-computer interface, as shown in Figure 1 The method comprises the following steps:

[0054] S1, data import and parameter setting, import the acquired SSVEP signal data, and set the response frequency, the window size of the local maximum value, and the offset value of the response frequency;

[0055] Specifically, in the present embodiment, the response frequency fr is set to [6.25, 7.75, 9.25, 10.75, 12.25].

[0056] S2, data denoising, time domain mean calculation is performed on the multi-channel acquired SSVEP signal to remove physiological artifacts;

[0057] S3, circularly processing each data block, including data preprocessing steps, baseline drift removal, notch filtering, band pass filtering, and fast Fourier transform are performed on each data block; the spectral data screening step is also included, the local maximum value of the spectral amplitude and the corresponding frequency are extracted within the set window, and the frequency that is consistent with the set response frequency and its second harmonic frequency and third harmonic frequency and within the allowed offset tolerance value is screened out;

[0058] Specifically, in the spectral data screening step, the local maximum value screening is included: for the extracted local maximum value, the frequency consistent with the characteristics of the SSVEP signal is screened out, which means that the selected signal frequency should be consistent with the set response frequency fr and its second harmonic frequency 2fr and third harmonic frequency 3fr, and the frequency selection condition is as follows: |f max -f r |<Δf, wherein Δf is the allowed offset tolerance value, by comparing the maximum amplitude at the response frequency and its harmonic frequency, only the frequency that meets the condition is retained.

[0059] Figure 2 The flowchart of the local maximum value screening is shown in FIG. 1, and the specific steps include: Figure 2

[0060] 21-Set variables: input window size and frequency (x) range;

[0061] 22-Check if the range is empty; if the range is empty, return “false”.

[0062] 23-Find the maximum value: find the maximum value in the given frequency range.

[0063] 24-If no maximum value is found, return “false”.

[0064] 25-Check if the range is empty (again): if the range is empty, return “false”.

[0065] 26-Save the array: save the frequency (x) and the corresponding maximum value (y) to the array.​

[0066] 27- Return array: return an array that holds the frequencies and the maxima.

[0067] Further, frequency filtering is included: for the extracted local maxima, filter out the frequencies that are consistent with the SSVEP signal characteristics, meaning that the selected signal frequency should be consistent with the set response frequency fr and its second harmonic frequency 2fr, third harmonic frequency 3fr, the frequency selection condition is as follows: |f max -f r | < Af, where Af is the allowed offset tolerance value, by comparing the maximum amplitude at the response frequency and its harmonic frequencies, only the frequencies that meet the conditions are retained.

[0068] Figure 3 The flow chart of the frequency filtering procedure is shown in FIG. 4, the specific steps include: Figure 3

[0069] 31- Set variables: input the response frequency (fre) and the offset value.

[0070] 32- Find the amplitude maximum value: find the amplitude maximum value (y_max) in the given frequency range.

[0071] 33- Output the corresponding frequency: output the frequency (x_max) corresponding to the amplitude maximum value.

[0072] 34- Check the frequency difference value: calculate the absolute value of the frequency difference value (|x_max-fre|).

[0073] 35- If the frequency difference value is less than the offset value, return "true".

[0074] 36- If the frequency difference value is greater than or equal to the offset value, return "false".

[0075] S4, result display, according to the filtering condition, output the corresponding information, if the filtering condition is met, output the data block that meets the condition and its characteristic frequency, and display the frequency spectrum diagram of all the data blocks that meet the condition; if not, output "no block found".

[0076] ​SSVEP refers to the electroencephalogram generated in the visual cortex when a subject gazes at a flickering visual stimulus modulated at a constant frequency. The frequency of the SSVEP is the same as the received visual stimulus and its harmonic response frequency. With this characteristic of SSVEP, a SSVEP-based BCI system can be designed, such as a SSVEP-based speller, in which different targets are encoded by different stimulus frequencies. When a subject needs to select a command, they can gaze at the corresponding flickering target stimulus on the interface that encodes the command. The generated SSVEP can be identified by a specially designed decoder, thereby obtaining the subject's intention. SSVEP has significant advantages: high signal strength, easy to detect and identify, high information transmission rate, short user training time, etc.

[0077] Signal processing is a crucial technology in BCI systems, involving how to extract effective information, i.e., electroencephalogram features, from raw electroencephalogram signals. However, in the data acquisition of SSVEP-based BCI systems, it is mainly affected by two aspects: on the one hand, the stimulus frequency of SSVEP itself is in the low frequency band (4-50HZ), which is easily disturbed by power frequency; on the other hand, physiological artifacts such as electrooculogram artifacts, electromyogram artifacts, electrocardiogram artifacts, body shaking, etc. Therefore, it is crucial to process the original SSVEP signal. At present, in the BCI system, the most widely used and effective signal processing methods mainly focus on time domain, frequency domain, and time-frequency domain.

[0078] From the time domain, signal processing techniques are used to quantify signal behavior by analyzing how data changes over time. This approach is particularly important in SSVEP signal processing, as SSVEP signals often need to be recorded and observed over long periods of time. To perform time-domain feature extraction, windowing and segmentation techniques are usually employed to divide the signal into multiple small segments. Local features within each window are extracted, allowing researchers to track changes in these features over different time periods. Since physiological signals often have nonlinear and non-stationary characteristics, windowing and segmentation are particularly crucial in processing such signals. Common SSVEP time-domain techniques include time-domain superposition, autoregressive modeling (AR), etc.

[0079] From the frequency domain, frequency-domain signal analysis focuses on observing the specific frequency components and distribution of SSVEP signals, which involves converting time-domain signals to the frequency domain for more in-depth analysis. In SSVEP signals, frequency-domain analysis can reveal the frequency distribution such as harmonic distribution and the contrast of specific frequencies such as response frequency. Common frequency-domain techniques include Fourier transform (FFT), power spectral density (PSD), filter banks, etc.

[0080] From the time-frequency domain, both the time domain and the frequency domain have their limitations. In order to fully reflect the dynamic changes of the signal, the time-frequency domain combines the spectral characteristics of the signal with the time domain characteristics for simultaneous analysis. Common EEG time-frequency feature extraction techniques include short-time Fourier transform, S-transform, matching pursuit, etc.

[0081] In the SSVEP-based BCI system, the quality of the SSVEP signal data and the feature extraction are crucial. Based on the characteristics of the consistency of the SSVEP signal frequency and the response frequency, the existing technology mainly focuses on developing algorithms in the frequency domain, mainly through analyzing the frequency characteristics of the SSVEP signal to extract the features of the signal, and then performing tasks such as classification, target recognition, encoding and decoding, etc.

[0082] The working principle of power spectral density analysis (PSDA): This method identifies the main frequency components of the signal by calculating the power spectral density of the SSVEP signal. It uses FFT (Fast Fourier Transform) to convert the time domain signal into a frequency domain signal, and induces a peak in the amplitude spectrum at the stimulation frequency of the SSVEP signal, which needs to be greater than the average amplitude of the lower and higher frequency bands. The specific process is to first search for the peak and determine the corresponding frequency, and secondly, to avoid false positives caused by spontaneous EEG signals, ensure that the normalized signal amplitude of the determined frequency is higher than the predefined threshold.

[0083] The calculation formula of the normalized amplitude spectrum is: Where X is the original signal data; FFT(X) represents the Fast Fourier Transform of the original signal; |FFT(X)| represents the modulus (amplitude) of FFT(x); ∑|FFT(x)| represents the sum of the amplitudes of all frequency components.

[0084] Main disadvantages and deficiencies: PSDA has low classification accuracy under conditions of strong noise interference or unstable signal. In addition, since this method only focuses on the power information of a single channel signal, it ignores the spatial dependence between different channel signals, and cannot fully utilize the signal information of multiple channels.

[0085] Working principle of canonical correlation analysis (CCA): The CCA method identifies the SSVEP signal by finding the maximum correlation between the EEG signal and the reference signal (such as sine and cosine waves), and it uses the spatial information of multiple channels to enhance the robustness of signal processing. The specific process of CCA is as follows: first, calculate the representative comprehensive index of the two groups of signals, then use the correlation coefficient of the two indexes to reflect the overall correlation of the two groups of signals. Finally, according to the maximum correlation coefficient, the corresponding stimulation frequency of the SSVEP signal is determined. The specific process is as follows: for the collected multi-channel EEG signal matrix X:

[0086]

[0087] where M is the number of leads, and N is the number of sampling points. The detailed steps of using CCA for EEG signal classification are as follows: first, use different integer multiple frequency sine functions to construct a template signal set Y:

[0088]

[0089] where f k is the stimulation frequency, N h is the total number of harmonics, N is the number of sampling points, and Fs is the sampling frequency. The core principle of CCA is to find two suitable linear transformations Wx and Wy, so that the correlation coefficient between one index x = X T W x and the other index y = Y T W y is maximum. Therefore, an optimization problem can be constructed to obtain the maximum correlation coefficient:

[0090]

[0091] where the maximum correlation coefficient between the collected multi-channel EEG signal X and the reference signal Y represents the corresponding stimulation frequency.

[0092] Disadvantages and deficiencies: the traditional CCA method does not consider the non-base frequency components of the signal, which may miss some important spectral information, resulting in a decrease in the robustness of classification.

[0093] Working principle of filter bank CCA (FBCCA): the FBCCA algorithm decomposes the SSVEP signal into multiple sub-components of different frequency bands, i.e. using a band-pass filter bank to decompose the input EEG signal into N sub-band components, and then performing the CCA algorithm on the N sub-band components. Then make a weighted average of the correlation coefficients obtained on the N sub-band, and get a overall correlation coefficient value corresponding to each stimulation frequency fk, k = 1, 2,..., S. Finally, select the maximum correlation coefficient value from the S correlation coefficient values, and the corresponding frequency is the final classification result.

[0094] The specific process is as follows: first, we use zero-phase I-type Chebyshev filter (other filters can also be used) to decompose the original EEG signal into N sub-band components, and then apply the standard CCA algorithm to each sub-band component to obtain the correlation coefficient value between each sub-band component and the pre-defined reference signal (corresponding to all stimulation frequencies). The correlation coefficient value corresponding to the Kth reference signal is represented by the vector p k , which contains N sub-band corresponding correlation coefficient values, as shown in the expression:

[0095]

[0096] Then we make a weighted square sum fusion of N sub-band components, that is

[0097]

[0098] Where the definition of the weighted function w(n) is:

[0099] w(n)=n -a +b,n∈[1,N]

[0100] Where a and b are constants, and their values are determined by the classifier performance to the best case.

[0101] Finally, we get S stimulus frequency corresponding to S weighted correlation coefficient, take the maximum corresponding to the stimulus frequency, that is, we identify the target frequency.

[0102] Main shortcomings and deficiencies: although FBCCA introduces harmonic components through filter bank to improve the recognition and classification efficiency, due to the large difference between different individuals, the generalization ability of different users still needs further verification.

[0103] The common shortcomings of the above algorithms are:

[0104] 1. Without considering the noise interference of power frequency interference and baseline offset.

[0105] 2. Usually in the form of "end to end", designed for specific task scenarios, such as classification, target recognition, coding and decoding, etc., resulting in the lack of flexibility of this kind of method, it is difficult to achieve the purpose of generalization.

[0106] 3. The original SSVEP signal data quality is not considered, and the preprocessing and screening steps are less. Artificial screening method is usually used to form the data set for algorithm processing.

[0107] The embodiment of the present application aims to solve the shortcomings of the existing SSVEP signal processing technology. The technical scheme of the present application proposes an efficient SSVEP signal processing and screening process to improve the accuracy and efficiency of signal recognition.

[0108] As Figure 4 The total scheme flow chart of a specific embodiment of the present application is shown, and the specific scheme is as follows:

[0109] 1. Data import and parameter setting:

[0110] First, the system imports the SSVEP signal data and sets the key parameters of the function, including the response frequency, the window size of the local maximum value, and the offset value of the response frequency. Among them, the response frequency f r is set to [6.25, 7.75, 9.25, 10.75, 12.25] in this embodiment.

[0111] 2. Data denoising and preprocessing:

[0112] The SSVEP signal collected by multiple channels is subjected to time domain mean calculation to remove physiological artifacts and introduce noise suppression. This process aims to improve the purity of the signal and provide an optimized initial signal environment for the next data processing.

[0113] 3. Loop processing each data block: the system processes each data block in a loop to ensure its integrity and accuracy:

[0114] 3.1 Data preprocessing as follows: baseline drift removal, notch filtering, band-pass filtering, and FFT transformation are performed on each data block in turn;

[0115] Baseline drift removal: a 3HZ low-pass filter is used to filter out low-frequency interference to stabilize the baseline.

[0116] Butterworth filter design is adopted:

[0117]

[0118] where H LP (s) represents the transfer function of the low-pass filter; s represents a complex variable used to describe the frequency characteristics of the system in Laplace transform; ω c represents the cutoff frequency of the low-pass filter, the cutoff frequency ω c = 2π × 3 rad / s; N represents the order of the filter, N = 1; a1, a1, …, a N represent the coefficients of the filter.

[0119] Notch filtering: a 50HZ 4th-order notch filter is used to remove power frequency interference and avoid interference from device noise such as power supplies and appliances.

[0120]

[0121] where H notch (s) represents the transfer function of the notch filter; s is a complex variable used to describe the frequency characteristics of the system; ω n represents the center frequency of the notch filter, the center frequency ω n = 2π × 50; ζ represents the damping ratio of the filter.

[0122] Band-pass filtering: 4th order band-pass filter with a frequency range from 4 Hz to 40 Hz. Filter out other noises except the fundamental and its harmonics, thus preserving the target frequency.

[0123]

[0124] where H bandpass (s) represents the transfer function of the band-pass filter; s is a complex variable used to describe the frequency characteristics of the system; ω l represents the low cut-off frequency of the band-pass filter, ω l = 2π x 4 rad / s; ω h represents the high cut-off frequency of the band-pass filter, ω h = 2π x 40 rad / s.

[0125] FFT transformation: Perform fast Fourier transform on the processed data to prepare for spectral analysis. The formula of the whole data preprocessing process is as follows:

[0126] y(n) = L -1 [L(x(n)) · H Lp (s) · H noth (s) · H bandpass (s)]

[0127] Y(k) = FFT(y(n)), k = 0, 1, 2, …, N-1

[0128] where x(n) is the input original signal sequence, n represents the discrete time point of the signal; y(n) is the signal sequence after a series of filtering processing; L(·) and L -1 (·) represent linear transformation and its inverse transformation between different domains of the signal, respectively; FFT(·) represents fast Fourier transform, which is used to convert the time domain signal y(n) to the frequency domain to obtain the frequency domain signal Y(k), where k represents the frequency index, and N is the length of the input signal.

[0129] As Figure 5 shown in the following figure are the time domain and frequency domain diagrams of data preprocessing.

[0130] 3.2 Spectrum data screening as follows:

[0131] Local maximum value screening: For spectral data, we set a fixed window size, and extract the local maximum value of amplitude and its corresponding frequency in each window. Assuming the window size is N, in the window Wi, we find:

[0132]

[0133] where A(f) is the spectral amplitude, f max and A maxThe frequency and the amplitude of the local maximum, respectively. This method can effectively reduce the calculation time and enhance the reliability of local peak identification.

[0134] Frequency screening: For the extracted local maximum, we screen the frequencies consistent with the characteristics of the SSVEP signal. This means that the selected signal frequency should match the set response frequency f r and its second and third harmonic frequencies (2f r , 3f r ). The frequency selection conditions are as follows:

[0135] |f max -f r |<Δf

[0136] Where Δf is the allowed offset tolerance. By comparing the maximum amplitude at the response frequency and its harmonic frequencies, we only keep the frequencies that meet the conditions.

[0137] 4. The results show

[0138] Finally, the system outputs the corresponding information according to the processing results. If the screening conditions are not met, it outputs "no block found", i.e. all experimental results are not good, if the conditions are met, it outputs the data block that meets the conditions and its characteristic frequency, and displays the spectrum of all data blocks that meet the conditions. For example Figure 6 is the screening result graph for a certain experiment.

[0139] By implementing the above SSVEP signal processing and screening scheme, not only the time required for data screening is greatly reduced, but also the quality of the signal is significantly improved, so that the experimenter can more effectively screen high-quality SSVEP data. For example Figure 7 is the comparison graph of the screening results before and after. This provides a reliable data set for subsequent machine learning and deep learning algorithms in tasks such as classification and target recognition. In addition, this scheme can also be used to check whether there are errors in the basic operations during data acquisition, thereby helping beginners to better learn and observe the characteristics of SSVEP signals. Through these advantages, the system provides strong support for the processing and analysis of SSVEP signals.

[0140] The technical scheme of the present application mainly includes data preprocessing and frequency screening. The data preprocessing is processed by time domain and frequency domain filters, which can effectively remove physiological artifacts and noise interference in the actual working environment, thereby improving the quality of the original SSVEP signal. The frequency screening step overcomes the shortcomings of time-consuming and laborious manual screening. According to the characteristics of SSVEP signals and the characteristics of the fundamental and harmonic waves, the function parameters are adjusted according to individual differences, and the effective SSVEP signal is quickly screened and retained, thereby providing strong support for subsequent tasks such as high-quality data sets.

[0141] Further referring to Figure 8 , as an implementation of the method shown in the above figures, the present application provides an embodiment of an apparatus, which corresponds to the method embodiment shown in Figure 1 , and the apparatus can be applied in various electronic devices.

[0142] In a second aspect, the present application also discloses an apparatus for SSVEP signal processing and screening based on brain-computer interface, as shown in Figure 8 , which comprises a data import module 81, a parameter setting module 82, a data denoising module 83, a data block processing module 84 and a result display module 85.

[0143] In one specific embodiment, the data import module 81 is configured to import the acquired SSVEP signal data; the parameter setting module 82 is configured to set the response frequency, the window size of the local maximum value and the offset value of the response frequency; and the data denoising module 83 is configured to perform time domain mean calculation on the SSVEP signal collected by multiple channels to remove physiological artifacts.

[0144] The data block processing module 84 is configured to cyclically process each data block, which comprises a data preprocessing submodule 841 configured to perform baseline drift removal, notch filtering, band-pass filtering and fast Fourier transform on each data block; and a spectral data screening submodule 842 configured to extract the local maximum value of the spectral amplitude and the corresponding frequency within the set window, and screen out the frequency that is consistent with the set response frequency and its second harmonic frequency and third harmonic frequency and within the allowed offset tolerance value.

[0145] The result display module 85 is configured to output the corresponding information according to the screening condition, and output the data block and its characteristic frequency that meet the screening condition and display the spectral graph of all the databases that meet the condition if the screening condition is met, and output “no block found” if the screening condition is not met.

[0146] The functions of the above modules correspond to the methods, which will not be described here.

[0147] The key innovations of the technical solutions of the present application mainly include three aspects: filter set, frequency domain screening, and signal processing and screening pre-system construction.

[0148] 1. Filter set: through the combination of low-pass filter, notch filter and band-pass filter, the problems of baseline offset, power frequency interference and noise interference in the SSVEP signal are simply and effectively solved, and the denoising function of the SSVEP signal data is completed.

[0149] 2. Frequency screening: through the combination of local maximum function and frequency screening function, the effective data of SSVEP signal in the experiment is further screened according to the characteristics of SSVEP signal, and the problem of time-consuming and laborious screening of data set is avoided. If only simple filter bank is used for signal denoising, the following problems cannot be solved: the specific response frequency is inconsistent with the current SSVEP signal frequency due to the factors such as the inattention of the experimenter, the shaking of the wearing device and the like; similarly, if only frequency screening is performed, the problems of environmental noise interference and baseline offset cannot be removed.

[0150] 3. Signal processing and screening pre-system construction: the existing SSVEP signal processing technology either does not screen the signal or is designed for a specific task such as classification and target recognition, and it is difficult to achieve commonality between different tasks. The technical solution of the present application completes a flexible and real-time SSVEP signal processing and screening system construction, so as to facilitate the completion of various task modules, such as providing high-quality data set, which is a plug-and-play technical solution.

[0151] Reference is made to Figure 9 , which shows a structural diagram of a computer device 900 of an electronic device suitable for implementing embodiments of the present application. Figure 9 The electronic device shown is only an example and should not impose any limitation on the functions and use range of embodiments of the present application.

[0152] As shown in Figure 9 , the computer device 900 includes a central processing unit (CPU) 901 and a graphics processor (GPU) 902, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 903 or programs loaded into a random access memory (RAM) 904 from a storage portion 909. In the RAM 904, various programs and data required for the operation of the device 900 are also stored. The CPU 901, the GPU 902, the ROM 903 and the RAM 904 are connected to each other through a bus 905. An input / output (I / O) interface 906 is also connected to the bus 905.

[0153] The following components are connected to the I / O interface 906: an input part 907 including a keyboard, a mouse, etc.; an output part 908 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 909 including a hard disk, etc.; and a communication part 910 including a network interface card such as a LAN card, a modem, etc. The communication part 910 performs communication processing via a network such as the Internet. A drive 911 can also be connected to the I / O interface 906 as necessary. A removable media 912 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 911 as necessary, so that a computer program read therefrom is installed in the storage part 909 as necessary.

[0154] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication part 910, and / or installed from the removable media 912. When the computer program is executed by the central processing unit (CPU) 901 and the graphics processor (GPU) 902, the above-described functions defined in the methods of the present disclosure are performed.

[0155] It should be noted that the computer-readable medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable medium may, for example, be— but is not limited to— an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the above. More specific examples of the computer-readable medium can include— but are not limited to— an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus, or device. In the present application, the computer-readable signal medium can include a data signal traveling in a baseband or an analog or digital carrier wave or other transport medium, in which the computer-readable program code is contained or stored. Such a program code traveling in a carrier wave can take any of a variety of forms, including but not limited to— electromagnetic, optical, or other types of signals. The computer-readable medium can also be any computer-readable medium other than the computer-readable medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The program code contained or stored by the computer-readable medium can be transmitted or propagated using any suitable medium, including but not limited to— wireless, wire line, optical fiber, RF, etc., or any suitable combination of the above.

[0156] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0157] The computer readable medium can be a storage medium or memory medium such as, but not limited to, readable and / or writeable nonvolatile memory devices (e.g., flash memory, etc.). However, a person of skill in the art will immediately appreciate that the application is not limited to these embodiments. The embodiments disclosed herein are intended to cover any and all modifications and alternatives within the scope of the present application. Further, it will be understood that various hardware and software components, as well as a combination of hardware and software components, can be utilized to implement the application. Accordingly, the application is not limited to a specific embodiment, but extends to whatever structural configuration enables the concepts described in this document to function. It will also be understood that whatever arrangement is utilized, it is designed and adapted to perform the functions set forth in the flow diagrams, flow charts, and / or block diagrams.

[0158] The modules described in the embodiments of the present application can be implemented by software, or by hardware. The described modules can also be arranged in a processor.

[0159] As another aspect, the present application also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method steps as described in the first aspect.

[0160] The above description is merely illustrative of the embodiments of the present application and the principles of the technology employed. It is understood that the scope of the present application is not limited to the specific combinations of technical features disclosed in the above embodiments, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features, without departing from the inventive concept. For example, the above technical features can be replaced with other technical features disclosed in the present application (but not limited to) having similar functions to form technical solutions.

Claims

1. A method for SSVEP signal processing and screening based on brain-computer interface, characterized in that, The method comprises the following steps: Data import and parameter setting, importing the acquired SSVEP signal data, and setting the response frequency, the window size of the local maximum value, and the offset value of the response frequency; Data denoising, time domain mean calculation is performed on the SSVEP signal collected by multiple channels to remove physiological artifacts; Each data block is processed in a loop, including a data preprocessing step, baseline drift removal, notch filtering, band pass filtering, and fast Fourier transform are performed on each data block; the spectrum data screening step is also included, the local maximum value of the spectrum amplitude and its corresponding frequency are extracted within the set window, and the frequency that is consistent with the set response frequency and its second harmonic frequency and third harmonic frequency and within the allowed offset tolerance value is screened out; The results show that the corresponding information is output according to the screening condition, if the screening condition is met, the data block and its characteristic frequency that meet the condition are output, and the spectrum graph of all the databases that meet the condition is displayed; if not, "no block found" is output.

2. The method for SSVEP signal processing and screening based on brain-computer interface according to claim 1, characterized in that, In the spectrum data screening step, local maximum value screening is included: For the spectrum data, a fixed window size is set, and the local maximum of the amplitude and its corresponding frequency are extracted in each window. The window size is N, and in the window Wi, find: where A(f) is the spectrum amplitude, f max and A max are the frequency and the amplitude of the local maximum, respectively.

3. The method for SSVEP signal processing and screening based on brain-computer interface according to claim 2, characterized in that, Frequency screening is also included: For the extracted local maximum value, the frequency consistent with the characteristics of the SSVEP signal is screened out, the selected signal frequency should be consistent with the set response frequency fr and its second harmonic frequency 2fr and third harmonic frequency 3fr, and the frequency selection condition is as follows: where Δf is the allowed offset tolerance value, by comparing the maximum amplitude at the response frequency and its harmonic frequencies, only the frequencies that meet the condition are retained.

4. The method for SSVEP signal processing and screening based on brain-computer interface according to claim 1, characterized in that, In the data preprocessing step, a low-pass filter with a cutoff frequency of 3HZ is used to filter out low-frequency interference to stabilize the baseline, and a Butterworth filter is designed, which is expressed by the following formula: ， wherein represents a transfer function of a low-pass filter; represents a complex variable, used in Laplace transform to describe the frequency characteristics of a system; represents a cut-off frequency of a low-pass filter, the cut-off frequency ; N represents an order of the filter, N = 1; represents a coefficient of the filter.

5. The method for SSVEP signal processing and screening based on brain-computer interface according to claim 1, characterized in that, In the data preprocessing step, a 4th-order notch filter with a center frequency of 50HZ is used to remove power frequency interference and avoid interference from device noise of power supply and appliances, which is expressed by the following formula: ， wherein represents a transfer function of the notch filter; is a complex variable used to describe the frequency characteristics of the system; represents a center frequency of the notch filter, the center frequency ; represents a damping ratio of the filter.

6. The method for SSVEP signal processing and screening based on brain-computer interface according to claim 1, characterized in that, In the data preprocessing step, the band pass filter is a 4th-order band pass filter with a frequency range from 4Hz to 40Hz, which filters out other noise except the fundamental wave and harmonic wave, thereby retaining the target frequency, which is expressed by the following formula: ， wherein represents a transfer function of a band-pass filter; is a complex variable for describing a frequency characteristic of a system; represents a low cut-off frequency of a band-pass filter, represents a high cut-off frequency of a band-pass filter, .

7. The method for SSVEP signal processing and screening based on brain-computer interface according to claim 1, characterized in that, In the data preprocessing step, the total process of data preprocessing is as follows: ， wherein is the input raw signal sequence, denotes the discrete time points of the signal; is the signal sequence after a series of filtering processes; denotes the linear transformation between different domains of the signal and its inverse transformation, respectively; denotes the fast Fourier transform, which is used to convert the time domain signal into the frequency domain to obtain the frequency domain signal wherein denotes the frequency index, and N is the length of the input signal; denotes the transfer function of the low-pass filter; denotes the transfer function of the notch filter; denotes the transfer function of the band-pass filter.

8. The method for SSVEP signal processing and screening based on brain-computer interface according to claim 1, characterized in that, The response frequency is set to [6.25, 7.75, 9.25, 10.75, 12.25].

9. A device for SSVEP signal processing and screening based on brain-computer interface, characterized in that, It comprises: A data import module configured to import the acquired SSVEP signal data; A parameter setting module configured to set the response frequency, the window size of the local maximum value, and the offset value of the response frequency; A data denoising module configured to perform time domain mean calculation on the SSVEP signal collected by multiple channels to remove physiological artifacts; A data block processing module configured to process each data block in a loop, including a data preprocessing submodule configured to perform baseline drift removal, notch filtering, band pass filtering, and fast Fourier transform on each data block; A spectrum data screening submodule configured to extract the local maximum value of the spectrum amplitude and its corresponding frequency within the set window, and screen out the frequency that is consistent with the set response frequency and its second harmonic frequency and third harmonic frequency and within the allowed offset tolerance value; The result display module is configured to output corresponding information according to the screening condition, output the data block and its characteristic frequency that meet the condition if the screening condition is met, and display the spectrum diagram of all the databases that meet the condition; and output "no block found" if the screening condition is not met.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by the processor, implements the method of any one of claims 1 to 8.

Images