Spatial coding and decoding method for steady-state visual evoked potential

Abstract

The invention discloses a space coding and decoding method for steady-state visual evoked potential, which realizes mapping and identification of a target and a space characteristic by combining a multi-frequency characteristic with a canonical correlation analysis method and utilizing a space coding characteristic of a stimulation target. By monitoring and analyzing SSVEP response distribution of surrounding targets, different central targets can be effectively distinguished, so that spatial coding is realized. In the decoding process, the target frequency corresponding to the most relevant frequency of the electroencephalogram signal is identified through the maximum correlation coefficient. Then, in combination with the frequency of the surrounding target, an inclusion index is calculated to further confirm the validity of the identification. And when the inclusion degree is greater than a set threshold value, considering that the current center target is identified as an effective center target. According to the method, the recognition precision can be improved, the electroencephalogram channel requirement is reduced, the information transmission rate and the real-time performance of the system are improved through the optimized space coding scheme, and the method is particularly suitable for low-delay and multi-target recognition brain-computer interaction tasks.

Description

Technical Field

[0001] The present invention relates to the fields of human-computer interaction, artificial intelligence, brain-computer interface technology, and in particular to a spatial encoding and decoding method based on steady-state visual evoked potential (SSVEP). Background Art

[0002] Brain-computer interface (BCI) technology involves the cross-integration of multiple disciplines. Among them, the brain-computer interface based on steady-state visual evoked potential (SSVEP-BCI) uses the human brain's specific response to specific visual frequencies, has the characteristics of directness and naturalness, and has great potential in future brain-computer interaction applications.

[0003] Although SSVEP-BCI technology has made significant progress. However, the steady-state visual stimulation system based on traditional liquid encoding still faces some technical bottlenecks and challenges. At present, the research on SSVEP-based brain-computer interface mainly focuses on frequency encoding or frequency-phase joint encoding. In the frequency encoding or frequency-phase encoding method, visual stimuli with different flickering frequencies or phase sequences must be assigned to each target to form a one-to-one stimulus-target pattern.

[0004] In the prior art, the spatial characteristics of visual stimulation and the contralateral effect of human optic nerve imaging are used to propose a method of decoding spatial information from the distribution of EEG activity topography on the scalp, which has a low degree of dependence on time or spectral patterns. Its main feature is the separation of target and stimulus, which can induce multiple targets with less stimulus. However, its defects are that it requires more EEG channels; the decoding results can only roughly reflect the direction observed by the human eye and cannot accurately locate; the information transmission rate is low, etc. Summary of the invention

[0005] Technical problem: In view of the above defects or deficiencies in the prior art, the present invention utilizes the multi-frequency processing mechanism of the human brain to propose a spatial encoding and decoding method for steady-state visual evoked potentials. The limited number of EEG signal channels can be used to complete the mapping of the spatial position information of the stimulus target and the spatial characteristics of the EEG signal, and complete the decoding and recognition to output the result of accurate positioning. The present invention aims to provide a spatial encoding and decoding method for SSVEP that requires few EEG channels, can accurately locate the target, and has a high information transmission rate.

[0006] Technical solution: To achieve the above purpose, the present invention adopts a spatial encoding and decoding method of steady-state visual evoked potential, which includes the following two parts: (I) A spatial coding method comprising the following steps: Step S1: The visual stimulus generation module generates spatially encoded stimulus targets, and the stimulus interface displays two groups of stimulus targets. Each group of stimulus targets consists of 9 targets in a 3×3 pattern, including 1 central target to be identified and 8 surrounding targets for providing spatial position information. Step S2: Set the frequencies of the central targets of the two groups of stimulation targets to the same frequency , and the frequencies of the surrounding targets are set to and , and the frequencies of the two groups of surrounding targets are required to be different; (ii) A spatial decoding method comprising the following steps: Step D1: The EEG signal acquisition module acquires the user's steady-state visual evoked potential (SSVEP) response signal, amplifies and converts the response signal into analog-to-digital form, and transmits the signal to the analysis and calculation module; Step D2: The analysis and calculation module preprocesses the collected EEG signal data, extracts features and completes recognition, and outputs the recognition result; Step D3: According to the recognition result, the operable device is controlled to perform subsequent operations.

[0007] in, In the step S1, the shape of the stimulus target is rectangular or elliptical, and the flashing mode of the stimulus is periodic flashing of sine wave, square wave or triangle wave, and the flashing frequency is 18-35 Hz.

[0008] The EEG signal acquisition module in step D1 adopts a non-invasive EEG acquisition device, using an 8-channel dry electrode or semi-dry electrode.

[0009] The signal transmission between the EEG signal acquisition module and the analysis and calculation module adopts optical fiber transmission in a wired transmission mode or Bluetooth / WIFI transmission in a wireless transmission mode.

[0010] The preprocessing of the collected EEG signal data in step D2 includes baseline drift removal and bandpass filtering to filter out frequency bands of non-concern and electrical oscillation frequencies. The cutoff frequency of the bandpass filter is 6 Hz to 90 Hz, and the notch width used in filtering is 1 Hz to 3 Hz.

[0011] The step D2 is specifically described as follows: after receiving the EEG signal, the analysis and calculation module first pre-processes the data, unpacks the data, converts the data formats of different packet sequences into EEG data in the shape of 8 channels, and stores the decoded data in a first-in-first-out buffer area. The data in the buffer area is processed to remove baseline drift and band-pass filtered to filter out frequency bands that are not of concern and filter out electrical oscillation frequencies; then, the pre-processed EEG signal data is extracted from the buffer area, and in the subsequent processing process, a reference template signal needs to be generated for the specified target frequency.

[0012] The analysis and calculation module adopts the canonical correlation analysis (CCA) method to extract frequency domain features related to the stimulation target frequency from the EEG signal and calculate the correlation coefficient of each target frequency.

[0013] The analysis and calculation module calculates the correlation coefficients of each frequency and identifies the frequency of the central target based on the matching degree between the maximum correlation coefficient and the remaining larger correlation coefficients and the frequencies of the surrounding targets.

[0014] The matching degree is to measure the degree of matching between the recognition result and the surrounding target frequency through the inclusion degree D. When the calculation of the inclusion degree D satisfies D>the set threshold, it is determined to be a valid recognition target.

[0015] The analysis and calculation module is implemented by any of the following computing devices: a general-purpose computer processor, a large-scale programmable logic device based on FPGA, or an embedded processing device.

[0016] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows: A spatial encoding and decoding method of steady-state visual evoked potential of the present invention has significant advantages over the prior art. By combining multi-frequency characteristics and canonical correlation analysis methods, the spatial encoding characteristics of the stimulus target are utilized to complete the mapping and identification of the target and spatial features. It is possible to improve the recognition accuracy of EEG signals, accurately distinguish central targets, reduce the demand for EEG channels, and complete high-precision target recognition with only 8 channels. Through the optimized spatial coding scheme, the present invention improves the information transmission rate, enhances the real-time performance and sensitivity of the system, and reduces the visual burden. While achieving precise spatial positioning, this method simplifies the system structure, reduces hardware requirements, and is convenient for practical application. It is particularly suitable for brain-computer interaction tasks that require high recognition accuracy and low latency, and can support complex multi-target recognition tasks. It has strong practicality and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The present invention is a flowchart of a SSVEP spatial encoding and decoding method provided by the present invention in one embodiment.

[0018] Figure 2 Schematic diagram of a stimulus target spatial encoding method of the present invention in one embodiment.

[0019] Figure 3 The following is a flowchart of the analysis and calculation module of the present invention in one embodiment.

[0020] Figure 4 A schematic diagram showing the generation of stimulus targets through a single spatial encoding.

[0021] Figure 5 A schematic diagram showing the results of the correlation coefficient analysis of the SSVEP signal evoked by the left central target.

[0022] Figure 6 A schematic diagram showing the results of the correlation coefficient analysis of the SSVEP signal evoked by the right central target. DETAILED DESCRIPTION

[0023] In the stimulation paradigm of the spatial encoding method, although the frequency information of the central target to be identified is the same, the user cannot distinguish the two central targets through their own frequency information. However, since the frequency information of the surrounding targets also enters the field of vision along with the central target, an SSVEP response is generated. Therefore, by monitoring and analyzing the SSVEP response distribution of the surrounding targets, the different central targets in the two groups of stimuli can be effectively distinguished, thereby achieving spatial encoding of the stimulus targets.

[0024] The present invention adopts a spatial encoding and decoding method of steady-state visual evoked potential, which includes the following two parts: (I) A spatial coding method comprising the following steps: Step S1: The visual stimulus generation module generates spatially encoded stimulus targets, and the stimulus interface displays two groups of stimulus targets. Each group of stimulus targets consists of 9 targets in a 3×3 pattern, including 1 central target to be identified and 8 surrounding targets for providing spatial position information. Step S2: Set the frequencies of the central targets of the two groups of stimulation targets to the same frequency , and the frequencies of the surrounding targets are set to and , and the frequencies of the two groups of surrounding targets are required to be different; (ii) A spatial decoding method comprising the following steps: Step D1: The EEG signal acquisition module acquires the user's steady-state visual evoked potential (SSVEP) response signal, amplifies and converts the response signal into analog-to-digital form, and transmits the signal to the analysis and calculation module; Step D2: The analysis and calculation module preprocesses the collected EEG signal data, extracts features and completes recognition, and outputs the recognition result; Step D3: According to the recognition result, the operable device is controlled to perform subsequent operations.

[0025] in, In the step S1, the shape of the stimulus target is rectangular or elliptical, and the flashing mode of the stimulus is periodic flashing of sine wave, square wave or triangle wave, and the flashing frequency is 18-35 Hz.

[0026] The EEG signal acquisition module in step D1 adopts a non-invasive EEG acquisition device, using an 8-channel dry electrode or semi-dry electrode.

[0027] The signal transmission between the EEG signal acquisition module and the analysis and calculation module adopts optical fiber transmission in a wired transmission mode or Bluetooth / WIFI transmission in a wireless transmission mode.

[0028] The data preprocessing in step D2 includes baseline drift removal and bandpass filtering to filter out frequency bands not of interest and electrical oscillation frequencies. The cutoff frequency of the bandpass filter is 6 Hz to 90 Hz, and the notch width used in filtering is 1 Hz to 3 Hz.

[0029] The specific description of step D2 is: after receiving the EEG signal, the analysis and calculation module first pre-processes the data, unpacks the data, converts the data format of different packet sequences into EEG data in the shape of 8 channels, and stores the decoded data in a first-in-first-out buffer. The data in the buffer is processed to remove baseline drift and band-pass filtered to filter out frequency bands that are not of interest and filter out electrical oscillation frequencies; then the pre-processed EEG signal data is extracted from the buffer. In the subsequent processing process, a reference template signal needs to be generated for the specified target frequency. Reference template signal matrix It can be generated by the following formula: , In this way, each target frequency is obtained The corresponding reference signal matrix, which contains the sine and cosine components of the fundamental frequency and its multiples, can reflect the characteristics of the frequency in the time domain and the frequency domain.

[0030] The feature extraction process of EEG signals first begins with extracting the EEG signal matrix With the reference template signal matrix The purpose of the comparison is to extract the frequency domain features related to each target frequency from the EEG signal. In order to evaluate the correlation between the EEG signal and the reference signal, canonical correlation analysis CCA is used. The purpose of CCA is to find the EEG signal matrix and the reference signal matrix The linear transformation between them maximizes the correlation between them.

[0031] In CCA analysis, the EEG signal matrix and the reference signal matrix The weight vectors are defined separately and , these two vectors linearly transform the signal. The goal of CCA is to maximize the correlation between the two. Through the CCA method, each target frequency can be calculated The corresponding correlation coefficient This correlation coefficient Represents the EEG signal at the target frequency For each target frequency, we can get a The value reflects the matching degree between the EEG signal and the frequency.

[0032] The last step is the classification and recognition process. In order to accurately identify the central target, it is not only necessary to rely on the maximum correlation coefficient, but also to consider the comprehensive performance of multiple correlation coefficients. Specifically, based on the correlation between the EEG signal and the reference signal matrix, the correlation coefficient sequence corresponding to each frequency is obtained. After calculation, the correlation coefficient sequence related to N target frequencies is finally obtained. Each of these Indicates The correlation between the target frequency and the EEG signal.

[0033] To identify the frequency of the central target , first determine the frequency that is most strongly correlated with the EEG signal, denoted as , the correlation coefficient corresponding to this frequency is During the recognition process, the frequency of the central target is most correlated with the EEG signal. ,therefore Should be the maximum correlation coefficient.

[0034] In order to further confirm the frequency of the central target and verify its relationship with the frequencies of the surrounding targets, a "containment degree" index is proposed to evaluate the matching degree of the frequencies of the surrounding targets. , further investigate the remaining 8 relatively large Value, that is , these frequencies correspond to the frequencies of the surrounding targets. The frequency set corresponding to these correlation coefficients is , then the inclusion degree Used to measure whether the frequencies corresponding to these correlation coefficients are consistent with the frequencies of surrounding targets.

[0035] Specifically, the inclusion It can be defined as the proportion of the identified correlation coefficients that match the frequencies of the surrounding targets. If these correlation coefficients correspond exactly to the frequencies of the surrounding targets, then the inclusion degree For incomplete matches, the inclusion will be less than 100%, and its effectiveness can be evaluated based on the set threshold.

[0036] The final target recognition judgment depends on the following two conditions: 1. Center frequency determination: First, check the frequency corresponding to the maximum correlation coefficient Is it consistent with the preset center target frequency? Conform 2. Inclusion evaluation: Calculate the inclusion If the value is greater than the set threshold (such as 75%), the currently identified central target is considered to be a valid identification target.

[0037] The cut-off frequency of the bandpass filter during the preprocessing operation of the analysis and calculation module is 6 Hz to 90 Hz. The filtered electrical oscillation frequencies are mainly 25 Hz and 50 Hz frequency components, and the notch width used during filtering is 1 Hz to 3 Hz.

[0038] The analysis and calculation module is a general-purpose computer processor or a computer device or embedded device based on large-scale programmable logic; Specific exemplary embodiments that can implement the present invention are described below by way of example. These embodiments are described in sufficient detail to enable those skilled in the art to practice the concepts disclosed in the present invention, and it should be understood that the various disclosed embodiments may be modified and other embodiments may be utilized without departing from the scope of the present disclosure. Therefore, the following detailed description is not to be considered restrictive.

[0039] The present invention aims to provide a spatial encoding and decoding method of SSVEP, which combines multi-frequency characteristics and typical correlation analysis methods, utilizes the spatial encoding characteristics of the stimulus target, and completes the mapping and recognition of the target and spatial features. It improves the feasibility of accurate spatial positioning of SSVEP and has strong practicality.

[0040] To achieve the above functions, the present invention adopts the following examples: Embodiment 1: See also Figure 1 The present invention provides a spatial coding and decoding method for SSVEP, which includes the following two parts: (I) Spatial coding method, see Figure 2 The following steps are involved: Step S1: The visual stimulus generation module generates spatially encoded stimulus targets, and the stimulus interface displays two groups of stimulus targets, each group of stimulus targets consists of 3×3, a total of 9 targets, including 1 central target to be identified and 8 surrounding targets for providing spatial position information; Step S2: Set the frequencies of the central targets of the two groups of stimulation targets to the same frequency , and the frequencies of the surrounding targets are set to and , and the frequencies of the two groups of surrounding targets are required to be different; (ii) A spatial decoding method comprising the following steps: Step D1: The EEG signal acquisition module acquires the user's steady-state visual evoked potential (SSVEP) response signal, amplifies and converts the response signal into analog-to-digital form, and transmits the signal to the analysis and calculation module; Step D2: The analysis and calculation module preprocesses the collected EEG signal data, extracts features and completes recognition, and outputs the recognition results; Step D3: Based on the recognition result, control the operable device to perform subsequent operations.

[0041] Embodiment 2: The SSVEP spatial coding method described in Example 1 has the following coding implementation principle: In such Figure 2 In the spatial encoding stimulation paradigm, although the frequency information of the central targets to be identified is the same, the user cannot distinguish the two central targets through their own frequency information. However, since the frequency information of the surrounding targets also enters the field of vision along with the central targets, SSVEP responses are generated. Therefore, by monitoring and analyzing the SSVEP response distribution of the surrounding targets, the different central targets in the two groups of stimuli can be effectively distinguished, thereby achieving spatial encoding of the stimulus targets.

[0042] Embodiment 3: The SSVEP spatial decoding method described in Example 1, wherein step D2 is specifically implemented as follows: See also Figure 3 After receiving the EEG signal, the analysis and calculation module first pre-processes the data and unpacks the data. The original EEG signal is unpacked into 8-channel EEG data in the data format of packet sequence input. The unpacked data is stored in a first-in-first-out buffer. The data in the buffer is processed to remove baseline drift and bandpass filtering, and the frequency bands that are not of concern are filtered out, and the oscillation frequency of electrical appliances is filtered out.

[0043] Then extract the preprocessed EEG signal data from the buffer. In the subsequent processing, it is necessary to generate a reference template signal for the specified target frequency. Reference signal matrix It can be generated by the following formula: , In this way, each target frequency is obtained The corresponding reference signal matrix, which contains the sine and cosine components of the fundamental frequency and its multiples, can reflect the characteristics of the frequency in the time domain and the frequency domain.

[0044] The feature extraction process of EEG signals first begins with extracting the EEG signal matrix With the reference template signal matrix The purpose of the comparison is to extract the frequency domain features related to each target frequency from the EEG signal. In order to evaluate the correlation between the EEG signal and the reference signal, canonical correlation analysis CCA is used. The purpose of CCA is to find the EEG signal matrix and the reference signal matrix The linear transformation between them maximizes the correlation between them.

[0045] See also Figure 4 , Figure 5 , Figure 6 , in CCA analysis, is the EEG signal matrix and the reference signal matrix Define the weight vectors separately and , these two vectors linearly transform the signal. The goal of CCA is to maximize the correlation between the two. Through the CCA method, each target frequency can be calculated The corresponding correlation coefficient This correlation coefficient Represents the EEG signal at the target frequency For each target frequency, we can get a The value reflects the matching degree between the EEG signal and the frequency.

[0046] The last step is the classification and recognition process. In order to accurately identify the central target, it is not only necessary to rely on the maximum correlation coefficient, but also to consider the comprehensive performance of multiple correlation coefficients. Specifically, based on the correlation between the EEG signal and the reference signal matrix, the correlation coefficient sequence corresponding to each frequency is obtained. After calculation, the correlation coefficient sequence related to N target frequencies is finally obtained. Each of these Indicates The correlation between the target frequency and the EEG signal.

[0047] To identify the frequency of the central target , first determine the frequency that is most strongly correlated with the EEG signal, denoted as , the correlation coefficient corresponding to this frequency is During the recognition process, the frequency of the central target is most correlated with the EEG signal. ,therefore Should be the maximum correlation coefficient.

[0048] In order to further confirm the frequency of the central target and verify its relationship with the frequencies of the surrounding targets, a "containment degree" index is proposed to evaluate the matching degree of the frequencies of the surrounding targets. , further investigate the remaining 8 relatively large Value, that is , these frequencies correspond to the frequencies of the surrounding targets. The frequency set corresponding to these correlation coefficients is , then the inclusion degree Used to measure whether the frequencies corresponding to these correlation coefficients are consistent with the frequencies of surrounding targets.

[0049] Specifically, the inclusion It can be defined as the proportion of the identified correlation coefficients that match the frequencies of the surrounding targets. If these correlation coefficients correspond exactly to the frequencies of the surrounding targets, then the inclusion degree 100%. For incomplete matches, the inclusion will be less than 100%, and its effectiveness can be evaluated based on the set threshold.

[0050] Embodiment 4: The classification and recognition process described in Example 3, the final target recognition judgment depends on the following two conditions: 1. Center frequency determination: First, check the frequency corresponding to the maximum correlation coefficient No and preset center target frequency Match.

[0051] 2. Inclusion evaluation: Calculate the inclusion If the value is greater than the set threshold (such as 75%), the currently identified central target is considered to be a valid identification target.

[0052] Embodiment 5: In the SSVEP spatial encoding method described in Example 1, the shape of the stimulus target in step S1 is rectangular or elliptical, the stimulus flickering mode is periodic flickering of sine wave, square wave or triangle wave, and the flickering frequency is 18-35 Hz.

[0053] Embodiment 6: In the SSVEP spatial decoding method described in Example 1, the signal transmission between the EEG signal acquisition module and the analysis and calculation module in step D1 is optical fiber transmission in a wired transmission mode or Bluetooth or WIFI transmission in a wireless transmission mode.

Claims

1. A method for spatial encoding and decoding of steady-state visual evoked potentials, characterized in that: It consists of the following two parts: (I) A spatial coding method comprising the following steps: Step S1: The visual stimulus generation module generates spatially encoded stimulus targets, and the stimulus interface displays two groups of stimulus targets. Each group of stimulus targets consists of 9 targets in a 3×3 pattern, including 1 central target to be identified and 8 surrounding targets for providing spatial position information. Step S2: Set the frequencies of the central targets of the two groups of stimulation targets to the same frequency , and the frequencies of the surrounding targets are set to and , and the frequencies of the two groups of surrounding targets are required to be different; (ii) A spatial decoding method comprising the following steps: Step D1: The EEG signal acquisition module acquires the user's steady-state visual evoked potential (SSVEP) response signal, amplifies and converts the response signal into analog-to-digital form, and transmits the signal to the analysis and calculation module; Step D2: The analysis and calculation module preprocesses the collected EEG signal data, extracts features and completes recognition, and outputs the recognition result; Step D3: According to the recognition result, the operable device is controlled to perform subsequent operations.

2. The method for spatial encoding and decoding of steady-state visual evoked potential according to claim 1, characterized in that: In the step S1, the shape of the stimulus target is rectangular or elliptical, and the flashing mode of the stimulus is periodic flashing of sine wave, square wave or triangle wave, and the flashing frequency is 18-35 Hz.

3. The spatial encoding and decoding method of steady-state visual evoked potential according to claim 1, characterized in that: The EEG signal acquisition module in step D1 adopts a non-invasive EEG acquisition device, using an 8-channel dry electrode or semi-dry electrode.

4. The spatial encoding and decoding method of steady-state visual evoked potential according to claim 3, characterized in that: The signal transmission between the EEG signal acquisition module and the analysis and calculation module adopts optical fiber transmission in a wired transmission mode or Bluetooth / WIFI transmission in a wireless transmission mode.

5. The spatial encoding and decoding method of steady-state visual evoked potential according to claim 1, characterized in that: The preprocessing of the collected EEG signal data in step D2 includes baseline drift removal and bandpass filtering to filter out frequency bands of non-concern and electrical oscillation frequencies. The cutoff frequency of the bandpass filter is 6 Hz to 90 Hz, and the notch width used in filtering is 1 Hz to 3 Hz.

6. The spatial encoding and decoding method of steady-state visual evoked potential according to claim 5, characterized in that: The step D2 is specifically described as follows: after receiving the EEG signal, the analysis and calculation module first pre-processes the data, unpacks the data, converts the data formats of different packet sequences into EEG data in the shape of 8 channels, and stores the decoded data in a first-in-first-out buffer area. The data in the buffer area is processed to remove baseline drift and band-pass filtered to filter out frequency bands that are not of concern and filter out electrical oscillation frequencies; then, the pre-processed EEG signal data is extracted from the buffer area, and in the subsequent processing process, a reference template signal needs to be generated for the specified target frequency.

7. The spatial encoding and decoding method of steady-state visual evoked potential according to claim 1, characterized in that: The analysis and calculation module adopts the canonical correlation analysis (CCA) method to extract frequency domain features related to the stimulation target frequency from the EEG signal and calculate the correlation coefficient of each target frequency.

8. The spatial encoding and decoding method of steady-state visual evoked potential according to claim 7, characterized in that: The analysis and calculation module calculates the correlation coefficients of each frequency and identifies the frequency of the central target based on the matching degree of the maximum correlation coefficient and the remaining correlation coefficient with the frequencies of the surrounding targets.

9. The method for spatial encoding and decoding of steady-state visual evoked potential according to claim 8, characterized in that: The matching degree is to measure the degree of matching between the recognition result and the surrounding target frequency through the inclusion degree D. When the calculation of the inclusion degree D satisfies D>the set threshold, it is determined to be a valid recognition target.

10. The spatial encoding and decoding method of steady-state visual evoked potential according to claim 7, characterized in that: The analysis and calculation module is implemented by any of the following computing devices: a general-purpose computer processor, a large-scale programmable logic device based on FPGA, or an embedded processing device.

Images